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Preface 


Perhaps no area of mathematics has changed as dramatically as matrices over 
the last 25 years. This is due to both the advent of the computer as well as the 
introduction and acceptance of matrix methods into other applied disciplines. 
Computers provide an efficient mechanism for doing iterative computations. This, 
in turn, has revolutionized the methods used for locating eigenvalues and 
eigenvectors and has altered the usefulness of many classical techniques, such as 
those for obtaining inverses and solving simultaneous equations. Relatively new 
fields, such as operations research, lean heavily on matrix algebra, while estab- 
lished fields, such as economics, probability, and differential equations, continue 
to expand their reliance on matrices for clarifying and simplifying complex 
concepts. 

This book is an algorithmic approach to matrix operations. The more 
complicated procedures are given as a series of steps which may be coded in a 
straightforward manner for computer implementation. The emphasis throughout 
is on computationally efficient methods. These should be of value to anyone who 
needs to apply matrix methods to his or her own work. 

The material in this book is self-contained; all concepts and procedures are 
stated directly in terms of matrix operations. There are no prerequisites for using 
most of this book other than a working knowledge of high school algebra. Some 
of the applications, however, do require additional expertise, but these are 
self-evident and are limited to short portions of the book. For example, elemen- 
tary calculus is needed for the material on differential equations. 

Each chapter of this book is divided into three sections. The first introduces 
concepts and methodology. The second section consists of completely worked-out 
problems which clarify the material presented in the first section and which, on 
occasion, also expand on that development. Finally, there is a section of problems 
with answers with which the reader can test his or her mastery of the subject 
matter. 

I wish to thank the many individuals who helped make this book a reality. I 
warmly acknowledge the contributions of William Anderson, whose comments on 
coverage and content were particularly valuable. 1 am also grateful to Howard 
Karp and Martha Kingsley for their suggestions and assistance. Particular thanks 
are due Edward Miliman for his splendid editing and support, David Beckwith of 
the Schaum staff for overseeing the entire project, and Marthe Grice for technical 
editing. 


RICHARD BRONSON 
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Chapter 1 


Basic Operations 


MATRICES 


A matrix is a rectangular array of elements arranged in horizontal rows and vertical columns, and 
usually enclosed in brackets. In this book, the elements of a matrix will almost always be numbers or 
functions of the variable t. A matrix is real-valued (or, simply, real) if all its elements are real 
numbers or real-valued functions; it is complex-valued if at least one element is a complex number or 
a complex-valued function. If all its elements are numbers, then a matrix is called a constant matrix. 


Example 1.1 
[3 i] 
34 
are all matrices. The first two on the left are real-valued, whereas the third is complex-valued (with i = V—1); 
the first and third are constant matrices, but the second is not constant. 


[us sinr rid] аё (-1.7, 2+ i6, -3i, 0] 


Matrices are designated by boldface uppercase letters. A general matrix A having r rows and c 
columns may be written 


where the elements of the matrix are double subscripted to denote location. By convention, the row 
index precedes the column index, thus, a;, represents the element of A appearing in the second row 
and fifth column, while a,, represents the element appearing in the third row and first column. A 
matrix А may also be denoted as [а„], where а, denotes the general element of A appearing in the 
ith row and jth column. 

A matrix having r rows and c columns has order (or size) “ғ by c," usually written r x c. The 
matrices in Example 1.1 have order 2 x 2, 2х3, and 1X4, respectively from left to right. Two 
matrices are equal if they have the same order and their corresponding elements are equal. 

The transpose of a matrix A, denoted as A’, is obtained by converting the rows of A into the 
columns of A’ one at a time in sequence. If A has order m X n, then A” has order n X m. 


Example 1.2 If 


VECTORS AND DOT PRODUCTS 


A vector is a matrix having either one row or one column. A matrix consisting of a single row is 
called a row vector; a matrix having a single column is called a column vector. The dot product А-В 
of two vectors of the same order is obtained by multiplying together corresponding elements of A 
and B and then summing the results. The dot product is a scalar, by which we mean it is of the same 
Beneral type as the elements themselves. (See Problem 1.1.) 


1 
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MATRIX ADDITION AND MATRIX SUBTRACTION 
The sum A+B of two matrices А = [a,] and B=(6,] having the same order is the matrix 
obtained by adding corresponding elements of A and B. That is, 
A +В = [а„] + [b;] = [2; + b,] 
Matrix addition is both associative and commutative. Thus, 
A+(B+C)=(A+B)+C and A+B=B+A 
(See Problem 1.2.) 
The matrix subtraction A — B is defined similarly: А and B must have the same order, and the 


subtractions must be performed on corresponding elements to yield the matrix [a; — Б]. (See 
Problem 1.3.) 


SCALAR MULTIPLICATION AND MATRIX MULTIPLICATION 


For any scalar k (in this book, usually a number or a function of t), the matrix kA (or, 
equivalently, Ak) is obtained by multiplying every element of A by the scalar k. That is, 
kA = k[aj] = [ka,]. (See Problem 1.3.) 

Let A = [2,] and B = [b,] have orders r X p and p X c, respectively, so that the number of 
columns of A equals the number of rows of B. Then the product AB is defined to be the matrix 
C = [c,] of order r x c whose elements are given by 


Р 
су= Жабу FEN FEL 


Each element c, of AB is а dot product; it is obtained by forming the transpose of the ith row of A 
and then taking its dot product with the jth column of B. (See Problems 1.4 through 1.7.) 

Matrix multiplication is associative and distributes over addition and subtraction; in general, it is 
not commutative. Thus, 


А(ВС) = (АВ)С  A(B*C)-AB«AC  (B-C)A-BA – CA 
but, in general, АВ # BA. Also, 
(AB)' = BTA" 


ROW-ECHELON FORM 


A zero row in a matrix is a row whose elements are all zero, and a nonzero row is one that 
contains at least one nonzero element. A matrix is a zero matrix, denoted 0, if it contains only zero 
rows. 

A matrix is in row-echelon form if it satisfies four conditions: 


(R1): All nonzero rows precede (that is, appear above) zero rows when both types are contained in 
the matrix. 


(R2) The first (leftmost) nonzero element of each nonzero row is unity. 


(R3): When the first nonzero element of a row appears in column c, then all elements in column c 
in succeeding rows are zero. 


(R4) The first nonzero element of any nonzero row appears in a later column (further to the right) 
than the first nonzero element of any preceding row. 
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Example 1.3 
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The matrix 


1.3 2:3 
00102 


00000 


satisfies all four conditions and so is in row-echelon form. (See Problems 1.11 to 1.15 and 1.18.) 


ELEMENTARY ROW AND COLUMN OPERATIONS 

There are three elementary row operations which may be used to transform a matrix into 
row-echelon form. The origins of these operations are discussed in Chapter 2; the operations 
themselves are: 


(El): Interchange any two rows. 


(E2): Multiply the elements of any row by a nonzero scalar. 


(E3) Add to any row, element by element, a scalar times the corresponding elements of another 


row. 


Three elementary column operations are defined analogously. 
An algorithm for using elementary row operations to transform a matrix into row-echelon form is 


as follows: 


STEP 1.1: 


STEP 1.2: 


STEP 1.3: 


STEP 1.4: 


STEP 1.5: 


STEP 1.6: 


STEP 1.7: 
STEP 1.8: 


Let R denote the work row, and initialize К = 1 (so that the top row is the first work 
том). 

Find the first column containing a nonzero element іп either гом R ог апу succeeding 
row. If no such column exists, stop; the transformation is complete. Otherwise, let C 
denote this column. 

Beginning with row R and continuing through successive rows, locate the first row 
having a nonzero element in column C. If this row is not row R, interchange it with row 
R (elementary row operation E1). Row R will now have a nonzero element in column 
C. This element is called the pivor; let P denote its value. 

If P is not 1, multiply the elements of row R by 1/P (elementary row operation E2); 
otherwise continue. 

Search all rows following row R for one having a nonzero element in column C. If no 
such row exists, go to Step 1.8; otherwise designate that row as row N, and the value of 
the nonzero element in row N and column C as V. 

Add to the elements of row N the scalar —V times the corresponding elements of row R 
(elementary row operation E3). 

Return to Step 1.5. 

Increase R by 1. If this new value of R is larger than the number of rows in the matrix, 
stop; the transformation is complete. Otherwise, return to Step 1.2. 


(See Problems 1.12 through 1.15.) 


RANK 


The rank (or row rank) of a matrix is the number of nonzero rows in the matrix after it has been 
transformed to row-echelon form via elementary row operations. (See Problems 1.16 and 1.17.) 
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Solved Problems 


L1 Find A-B and B-C” for 


5 7 
se] fajo *6(-8) + (~7)(-9) = 50 
- -9 


1.2 Show that A +B=B +A for 
[iom B-f 2] 
vae [: sleli 2]-[ в э+съ] [в а] 
вел s] [o 3] Гаа ase d 
Since the resulting matrices have the same order and all corresponding elements are equal, А + В = 


Bta. 


1.3 Find ЗА — 0.5B for the matrices of Problem 1.2. 


34 - osa -i[ ros J- [9 x] [Seu 0.5(5) 


32) (3)) 7]0.5(6) 0.5(-7) 
= [22 3'= 2:8 -[2 0.5 
(46-3 9-(35] 71 3 23] 


L4 Find AB and BA for the matrices of Problem 1.2. 


эв=[% 1][& 3]- [39:59 253-0] [26 <i 
вл- [5 516 = КИЕ «2 C5) a E 


Note that, for these matrices, АВ # BA. 


LS Find AB and BA for : 
afl 2:3 [7 8 

Ac Я 58] а Bf | 

Since A has three columns while В has only two rows, the matrix product AB is not defined. But 

K HIP 2 3]-[ 7(1) + 8(4) 7(2) + 8(-5) 7(3) + &6) 

0 -9/[4 -5 6) [0(1)+(—9)(4) 0(2)+(-9)(-5) 0(3)*(-9)(6) 


-[ 39 -26 ө] 
-36 45 -54 


ВА = 
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1.6 


17 


1.8 


Verify that (BA)" = АВ” for the matrices of Problem 1.5. 
1 412 o 1(7) + 4(8) 1(0) + 4(-9) 39 ~36 
[: -sI %]+|2@)+‹—5ув) 20)+ 5-9) |=] -26 as 
3 7 3(7) + 6(8) 3(0) + 6(-9) 69 —54 
which is the transpose of the product BA found in Problem 1.5. 


A'B 


Find AB and AC if 


4 20 2. 9. 4 31 -3 
A-| 2 10 B=]. 2-0 -2 c=| 02 6 
-2 c та 1 5142: 1 


4(2) + 2(2) + 0(-1) 4(3) + 2(-2) + 0(2) 4(1) + 2(-2) + 0(1) 
AB-| 2(2*1)*0(-1) 2(3) + 1(-2) + 02) 2(1) + 1(-2) + 0(1) 
=2(2) + (—1)(2) + 10-1) -2()*(-1)(-2)*10) -20)* (C1(72) + 1) 


о 80 
=| 6 40 
-7 -2 1 


4(3) + 2(0) + ((-1) 4(1) + 22) + 0(2) 4(—3) + 2(6) + 0(1) 12 8 0 
2(3) + 1(0) + (1) 2(1) + 1(2) + 0(2) 2(—3) + 1(6) + 0(1) -| 6 4 e] 
~2(3) + (-1)00) + 10-1) =2(1) +(-1)02) + 102) 72(73) + (71)(6) + 1(1) ep 0 1 


AC= 


Note that, for these matrices, АВ = AC and yet B # C. This shows that the cancellation law is not valid 
for matrix multiplication. 


A matrix is partitioned if it is divided into smaller matrices by horizontal or vertical lines 
drawn between entire rows and columns. Determine three partitionings of the matrix 


LG 3 $ 
A=|0 0 5 6 
7 8 -1 -2 


There аге 2° – 1 = 31 different ways in which А can be partitioned with at least one partitioning line. 
By placing a line between each two rows and each two columns, we divide A into twelve 1 x 1 matrices, 
obtaining 


By placing one line between the first and second rows and another line between the second and third 
columns, we construct the partitioning 


2 _[B C 
a HE F 
h 1 &U-Lb 2 
where 
00 5 6 
вер) сера в [99] к-[ 1 5] 


A third partitioning can be constructed by placing a single line between the third and fourth columns 
of A. Then А = [G, Н], where 
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t2 3 4 
G=|0 0 5 and  H-| 6 
78 -1 -2 
A partitioned matrix can be viewed as a matrix whose elements are themselves matrices. 


1.9 The arithmetic operations defined above for matrices having scalar elements apply as well to 
partitioned matrices. Determine AB and A — B if 


-[ 5] 


А= [Е c 


FG 
and в=[Р Е 


©] э бер eld) en 7) 


3 4 00 


^n - [Ct *br CG + DE 
“LEF+CF ЕС + СЕ 


КЕНЕ 


[13-36 


" 


10 8 15 13 
14 2 3 3 


E т 


1.10 Partitioning can be used to check matrix multiplication. Suppose the product AB is to be 
found and checked. Then A and B are replaced by two larger partitioned matrices, such that 
their product is 


(в/в) [ав кс 


where R is а new row consisting of the column sums of A, and С is a new column consisting of 
the row sums of B. The resulting matrix has the original product AB in the upper left 
partition. If no errors have been made, AC consists of the row sums of AB; RB consists of the 
column sums of AB; and RC is the sum of the elements of AC as well as the sum of the 
elements of RB. Use this procedure to obtain the product 


[; з 2 2) 
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1.11 


1.13 


We form the partitioned matrices and find their product: 


13 ; 
3 cp dia 
ШЕЕ 22! 


The product AB is the upper left part of the resulting matrix. Since the row sums of this product, the 
column sums, and their sums are correctly given in the matrix, the multiplication checks, 


Determine which of the following matrices are in row-echelon form: 


1.:2- t0 
aoo a] вр] с-[ i 
00 00 
0104 
p-[o o 1 3] в=[ 3 3] 
0001 


Only A and D are in row-echelon form. B is not, because the first (leftmost) nonzero element in the 
second row is further left than the first nonzero element in the top row, violating condition R4. 
Condition R2 is violated in the first row of C. Matrix E violates condition R3, because the first nonzero 
element in the lower row appears in the same column as the first nonzero element of the upper row. 


Use elementary row operations to transform matrices B, C, and E of 
Problem 1.11 into row-echelon form. 


We follow Steps 1.1 through 1.8 in each case, but for simplicity list only those steps that result in a 
matrix manipulation. For B, with R=1 (Step 1.1) and C=1 (Step 1.2), we apply Step 1.3 and 
interchange rows 1 and 2, obtaining. 


ШЕ 


which is in row-echelon form. For matrix C, with R = 1 (Step 1.1), C=2 (Step 1.2), and P =2 (Step 
1.3), we apply Step 1.4 and multiply all elements in the first row by 1/2, obtaining 


Io $3 
001 


which is in row-echelon form. For matrix E, with R = 1 (Step 1.1), C = 1 (Step 1.2), and N = 2and V - 4 
(Step 1.5), we apply Step 1.6 by adding, to each element in row 2, —4 times the corresponding element 
in row 1; the result is 


1 2 3 12 3 
mem 9+(-4)(2) т+сөв]=[& 1 4) 


which is in row-echelon form. 


Transform the following matrix into row-echelon form: 
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® эй 6 
3 8 9 10 
2 so Же 


Here (and in later problems) we shall use an arrow to indicate the row that results from each 
elementary row operation. 


э|0 2 12 -8 and V=3: Add –3 times the first 


1 2 =f : Step 1.6 with А = 1, C=1, N72, 
12 Lb 32 -3 row to the second row. 


and V-2: Add -2 times the first 
row to the third row. 


E | Step 1.6 with R=1, C21, N=3, 


P=2: Multiply the second row 


[ 2-1 | Step 1.4 with R=2, C=2, and 
by 1/2. 


L2 =) | Step 1.6 with R -2, C22, N=3, 

01 6 -4 and V- -5: Add 5 times the 
—10 0 34 -34 Second row to the third row. 

422^. Ж Step 1.4 with R 73, C - 3, and 

01 6 5| Р = 34: Multiply the third row by 
10 0 Lu 1/34. 


1.14 Transform the following matrix into row-echelon form: 


2 105 
3 6 11 
«у 1:58, 


—[1 1/2 0 5/2] Step 1.4 with R=1, C - 1, and 
3 6 тї P-2: Multiply the first row by 
5 Drs 1/2. 


11/20 5/2 Step 1.6 with R=1, C=1, N=2, 
|0 9/2 1 -13/72 and V-3: Add —3 times the first 

$ 7 | 8 row to the second row. 

1 12 0. SR Step 1.6 with К = 1, C21, N=3, 

0 9/2 1 -13/2 and V—5: Add —5 times the first 
—LO 9/2 1 -9/2 row to the third row. 


12 0 5/2 Step 1.4 with R=2, C=2, and 
P=9/2: Multiply the second row 
by 2/9. 

12 0 5/2 Step 1.6 with R=2, C22, N=3, 

1 2/9 -13/9 and V=9/2: Add —9/2 times the 

0 0 2 


second row to the third row. 


1 2/9 -13/9 P-2: Multiply the third row by 


1 
0 
0 
112 0 | Step 1.4 with R - 3, С= 4, and 
0 
оо о 1 12. 
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1.15 Transform the following matrix into row-echelon form: 


1.16 


1.7 


1 2/3 1/3 -4/3 1/3] Step 1.4: Multiply the first row 
23 0 -1 -i| by 1/3. 
1-6 3 -8 7 


1 2/3 1/3 -4/ 1/3 Step 1.6: Add —2 times the first 
0 5/3 -2/3 5/3 -51 row to the second row. 
1 -6 t] -8 7 


1 2/3 1/3 -4/ 1/3 Step 1.6: Add —1 times the first 
0 53 =2/3 5/3 —5/3 row to the third row. 
0 -20/3 8/3 -20/3 20/3 


1 2/3 13 -4/3 1/3 Step 1.4: Multiply the second row 
0 1 -2/5 1 zn | һу 3/5. 
0 -20/3 8/3 -20/3 20/3 
[| 213 1/3 -4/3 EI Step 1.6: Add 20/3 times the 
0 1 -2/5 1 =I second row to the third row. 
—lo 0 0 0 0 


Determine the rank of the matrix of Problem 1.14. 


Because the row-echelon form of this matrix has three nonzero rows, the rank of the original matrix 
is 3. 


Determine the rank of the matrix of Problem 1.15. 


Because the row-echelon form of this matrix has two nonzero rows, the rank of the original matrix 
is 2. 


Show that row-echelon form is not unique if a matrix has rank 2 or greater. 


Such a matrix has at least two nonzero rows after it is transformed into row-echelon form. By 
adding the second row to the first row, a different row-echelon-form matrix is produced. As an example. 
if we add the second row to the first row of the row-echelon-form matrix obtained in Problem 1.14, we 
obtain 


0 1 2/9 -13/9 


[e 3/2 2/9 "sl 
0 0 0 1 


Which is also in row-echelon form. 


10 
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Supplementary Problems 


In Problems 1.19 through 1.32 let 


ia Ra efh it] ofita G] ena a 


1.19 


1.20 


1.32 


1.33 


bom 663 3 


Find (a) А + B; (b) ЗА; (c) 2A – 3B; (d) € - D; and (e) А + Е. 


Designate the columns of A as A, and A,, and the columns of C as C,,C,, and C,, from left to right. 
Then calculate (a) A, * À;; (b) C,* C;; and (с) C,*C,. 


Find (a) AB; (b) BA; (c) (AB)'; (d) B'A'; and (e) A'B*. 
Find (a) CD and (b) DC. 
Find A(A + В). 
Find (a) CE and (b) EC. 
Find (a) CF and (5) FC. 
Find (a) EF and (b) FE. 
Transform A to row-echelon form. 
Transform B to row-echelon form. 
Transform C to row-echelon form. 
Transform D to row-echelon form. 
Transform E to row-echelon form. 
Find the rank of (a) A; (b) В; (c) C; (d) D; and (e) E. 
Find two matrices, neither of which is a zero matrix, whose product is a zero matrix. 
The price schedule for a New York to Miami flight is given by the vector P — (240, 180, 89], where the 
elements denote the costs of first class, business class, and tourist class tickets, respectively. The number 
of tickets of each class purchased for a particular flight is given by the vector N = [8, 21, 115). What is the 
significance of P*N? 
The inventory of computers at each outlet of a three-store chain is given by the matrix 
| 9 H 
№= |15 4 
7 0 


where the rows pertain to the different stores and the columns denote the number of brand X and brand 
Y computers, respectively, in each store. The wholesale costs of these computers are given by the vector 
D = (700, 1200]". Calculate ND and state its significance. 


Chapter 2 


Simultaneous Linear Equations 


CONSISTENCY 
A system of simultaneous linear equations is a set of equations of the form 


Gy Xp + ах, + ax +t aux, = b, 
ах, + ах, dx, + + aux mb, 


QU. Qux tux +7 


(21) 


The coefficients ay (= 1, ..m; j-1,2,...,n) and the quantities b, (i7 1,2,..., m) are 
known constants. The x, ( 1,2,...,n) are the unknowns whose values are sought. 

A solution for system (2.1) is a set of values, one for each unknown, that, when substituted in 
the system, renders all its equations valid. (See Problem 2.1.) A system of simultaneous linear 
equations may possess no solutions, exactly one solution, or more than one solution. 


tax, = b 


Example 2.1 The system 
x,*x,-71 
x, +x,=0 


has no solutions, because there are no values for x, and x, that sum to 1 and 0 simultaneously. The system 


x,*x2l 
x,*2x, 72 


has the single solution x, = 0, x, = 1; and 


хх; 
2x, ~2x, =0 


has a solution, x, = x; for every value of x,. 


A set of simultaneous equations is consistent if it possesses at least one solution; otherwise it is 
inconsistent, 


MATRIX NOTATION 
System (2.1) is algebraically equivalent to the matrix equation 


AX-B (22) 
а, ауса, fi b, 
an аса, x b. 
where MEM 3 Х=|. : B-|. Ы 
Ami Amz 83774, Xn bm 


The matrix A is called the coefficient matrix, because it contains the coefficients of the unknowns. 
The ith row of A (i=1,2,...,/) corresponds to the ith equation in system (2.1), while the jth 
column of A (j=1,2,..., п) contains all the coefficients of х,, one coefficient for each equation. 

The augmented matrix corresponding to system (2.1) is the partitioned matrix [A | B]. (See 
Problems 2.2 through 2.4.) 


1 
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THEORY OF SOLUTIONS 
Theorem 2.1: The system AX = B is consistent if and only if the rank of A equals the rank of [A | B]. 


Theorem 2.2: Denote the rank of A as К, and the number of unknowns as л. If the system АХ = B is 
consistent, then the solution contains n — k arbitrary scalars. 


(See Problems 2.5 to 2.7.) 

System (2.1) is said to be homogeneous if B = 0; that is, if b, = b; =+-- = 6,,=0. If B#0 [i.e., 
if at least опе b, (= 1,2,..., m) is not zero], the system is nonhomogeneous. Homogeneous 
systems are consistent and admit the solution x, = x,- x, = 0, which is called the trivial solution; 
a nontrivial solution is one that contains at least one nonzero value. 


Theorem 2.3: Denote the rank of A as k, and the number of unknowns as n. The homogeneous 
system АХ = 0 has a nontrivial solution if and only if n # К. (See Problem 2.7.) 


SIMPLIFYING OPERATIONS 


Three operations that alter the form of a system of simultaneous linear equations but do not alter 
its solution set are: 


(O1) Interchanging the sequence of two equations. 
(02) Multiplying an equation by a nonzero scalar. 
(03) Adding to one equation a scalar times another equation. 


Applying operations O1, O2, and O3 to system (2.1) is equivalent to applying the elementary 
row operations El, E2, and ЕЗ (see Chapter 1) to the augmented matrix associated with that system. 
Gaussian elimination is an algorithm for applying these operations systematically, to obtain a set of 
equations that is easy to analyze for consistency and easy to solve if it is consistent. 


GAUSSIAN ELIMINATION ALGORITHM 


STEP 2.1: Form the augmented matrix [A | B] associated with the given system of equations. 

STEP 2.2: Use elementary row operations to transform [A | B] into row-echelon form (see Chapter 
1). Denote the result as (C | DJ. 

STEP 2.3: Determine the ranks of C and [C | D]. If these ranks are equal, comtinue; the system is 
consistent (by Theorem 2.1). If not, stop; the original system has no solution. 

STEP 2.4: Consider the system of equations corresponding to [C | D], discarding any identically 
zero equations. (If the rank of С is k and the number of unknowns is п, there will be 
n — k such equations.) Solve each equation for its first (lowest indexed) variable having 
a nonzero coefficient. 

STEP 2.5: Апу variable not appearing on the left side of any equation is arbitrary. All other 
variables can be determined uniquely in terms of the arbitrary variables by back 
substitution. 


(See Problems 2.5 through 2.8.) Other solution procedures are discussed in Chapters 3, 4, 5, and 21. 


PIVOTING STRATEGIES 


Errors due to rounding can become a problem in Gaussian elimination. To minimize the effect of 
roundoff errors, a variety of pivoting strategies have been proposed, each modifying Step 1.3 of the 
algorithm given in Chapter 1. Pivoting strategies are merely criteria for choosing the pivot element. 
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Partial pivoting involves searching the work column of the augmented matrix for the largest 
element in absolute value appearing in the current work row or a succeeding row. That element 
becomes the new pivot. To use partial pivoting, replace Step 1.3 of the algorithm for transforming a 
matrix to row-echelon form with the following: 


STEP 1.3’: Beginning with row R and continuing through successive rows, locate the largest 
element in absolute value appearing in work column C. Denote the first row in which 
this element appears as row /. If / is different from R, interchange rows / and R 
(elementary row operation E1). Row R will now have, in column C, the largest 
nonzero element in absolute value appearing in column C of row R or any row 
succeeding it. This element in row R and column C is called the pivor; let P denote its 
value. 


(See Problems 2.9 and 2.10.) 


Two other pivoting strategies are described in Problems 2.11 and 2.12; they are successively 
more powerful but require additional computations. Since the goal is to avoid significant roundoff 
error, it is not necessary to find the best pivot element at each stage, but rather to avoid bad ones. 
Thus, partial pivoting is the strategy most often implemented. 


Solved Problems 


2.1 Determine whether x, 72, x, = 1, and x, = —11 is a solution set for the system 


2x, * х, =5 
3x, + 6x, + x, 


Sx, + 7х, + ху=8 


Substituting the proposed values for the unknowns into the left side of each equation gives 
2(2) + 11) =5 
3(2) + 6(1) + 1(-11 
5(2) + 7(1) + 1(-11) =6 
The last equation does not yield 8 as required; hence the proposed values do not constitute a solution 
set. 


2.2 Write the system of equations given in Problem 2.1 as a matrix equation, and then determine 
its associated augmented matrix. 


2 y 0 Ы 5 
A=]3 6 1 X-2|n B-|1 
5-7 1 Xs, 8 


The original system can be written as AX — B; its augmented matrix is 


z rus 
[А18] = [3 6 
5 7 
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2.3 Write the following system of equations in matrix form, and then determine its augmented 
matrix: 


3x,+2x,+ x,- 4x,- 1 
2x, + 3x, Xs 
x, — 6x, + 3x, = 8x, 


This system is equivalent to the matrix equation 
х, 
з 21-4]. 1 
2 
2 зо -1|,2 1-1-1 
5 
l 63 x8 x, 7 
The associated augmented matrix is 
3 2 1 see 1 
[А|в]=|2 3 0 -1i-1 
Lcd BS i 
Observe that in both A and [A | B], the zero in the second row and third column corresponds to the 
zero coefficient of x, in the second equation of the original system. 


2.4 Write the set of simultaneous equations that corresponds to the augmented matrix 
1 2/3 1/3 -4/3:1/3 
[А|в]=|0 1 -2/5 1 i-1 
0 0 0 0 0 
The corresponding set of equations is 


хойуу iret 
nint n=l 


The third equation reduces to the tautology 0 = 0 and is not written. Nor do we write any variable 
having a zero coefficient. 


2.5 Solve the set of equations given in Problem 2.1 by Gaussian elimination. 


The augmented matrix for this system was determined in Problem 2.2 to be 
21 0:5 
[AlB]2]3 6 1:1 
5 7 118 
Using the results of Problem 1.14, we transform this matrix into the row-echelon form 


112 0152 
[c|D]-|o 1 2/9:-13/9 


0 Q0 Of 1 


It follows from Problem 1.16 that the rank of [C | D] is 3. Submatrix C is also in row-echelon form 
and has rank 2. Since the rank of С does not equal the rank of [C | D], the original set of equations is 
inconsistent. The problem is the last equation associated with [C | D], which is 

Or, + Ox, + 0x, 71 


and which clearly has no solution. 
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2.6 


27 


Solve the set of equations given in Problem 2.3 by Gaussian elimination. 


The augmented matrix for this system was determined in Problem 2.3 to be 


3 21-411 
[AlB]-|2 3 0 -1:-1 
I -6 3 -81 7 
Using the results of Problem 1.15, we transform this matrix into the row-echelon form 
1 2/3 Y3 -4/113 
[C|ID]-]O 1 -2/5 1! i-1 
00 0 ото 
It follows from Problem 1.17 that the rank of (C | D] is 2. Submatrix C is also in row-echelon form, and it 


also has rank 2. Thus, the original set of equations is consistent. 
Now, using the results of Problem 2.4, we write 


x, + dx, + 0х 3х, 4 
х,-{ху+ х,=-1 


as the set of equations associated with [C | DJ. Solving the first equation for x, and the second for x,, we 
get 

х=} бх. 1х + бх, 

х= -1+ dx,- x, 


Since x, and x, do not appear оп the left side of any equation, they are arbitrary. The unknown x, is 


completely determined in terms of the arbitrary unknowns. Substituting it into the first equation, we 
calculate 


x= 4-4-1 + руду) day + Sy 
=1~ jx, +2x, 
The complete solution to the original set of equations is 
x,=1- ix, + 2x, 
x,7-ltdix-x, 


with x, and x, arbitrary. 


Solve the following set of homogeneous equations by Gaussian elimination: 
7x, +9x,=0 
2x,+ x- x,-0 
Sx, + 6х, +2x,;=0 


By converting this system to augmented-matrix form and then transforming the matrix into 
row-echelon form (Steps 1.1 through 1.8), we get 


07 
Ер 1 as] 
56 210 


12 -Af 


1 
кше t y 
0 0 


16 
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The rank of the coefficient matrix A is thus 2, and because there are three unknowns in the original set 
of equations, the system has nontrivial solutions. The set of equations associated with the augmented 
matrix [C | D] is 


x,*ix-in-20 


x, + $x,=0 
0-0 
Solving for the first variable in each equation with a nonzero coefficient, we obtain 
x--h*in 
х= — Fay 


Therefore, x, is arbitrary. Solving for x, and x, in terms of x, by back substitution, we find 
x, = — 5x, 
x)= 10б) + by = Bay 


Solve the following set of equations: 
xX, +2x,- х= 6 
3x, + 8x, 4 9x, 10 
2x, ~ x,*t2x,-2 -2 
The augmented matrix associated with this system is 
1 2,4 
[AlB]-]|3 8 9 
= ES 


which, in Problem 1.13, was transformed into the row-echelon form 


12-116 
[c|D]=]0 1 6:-4 
00 ti-t 


Both С and [C | D] have rank three, so the system is consistent. The set of equations associated with this 
augmented matrix is 
x,+2x,- х= 6 
x, + 6x,=—4 
z=- 


Solving each equation for the first variable with a nonzero coefficient, we obtain the system 
x= 6-2x, +4, 


x)= -4- 6x, 
-1 


ху 


which can be solved easily by back substitution beginning with the last equation. The solution to this 
system and to the original set of equations is x, = 1, x, = 2, and x, = - 1. 


Solve the following set of equations by (а) standard Gaussian elimination and (b) Gaussian 
elimination with partial pivoting, rounding all computations to four significant figures: 


0.00001x, + x, = 1.00001 
хү+х,=2 
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(a) We write the system in matrix form, rounding 1.00001 to 1.000. Then we transform the augmented 
matrix into row-echelon form using the algorithm of Chapter 1, in the following steps: 


|н 1 ! 1.000 | 
1 1 i 2 
-[ 1 100000 | kw. 
[ 1 1 | 2 
1 100000 100000 
al 0 —-100000 | end 
1 100000 | 100000 
EI 0 pcd. 4 | 


(Note that we round to —100000 twice in the next-to-last step.) The resulting augmented 
matrix shows that the system is consistent. The equations associated with this matrix are 


x, + 100000x, = 100000 
х,=1 


which have the solution x, = 0 and х, = 1. However, substitution into the original equations 
shows that this is not the solution to the original system. 


(b) Transforming the augmented matrix into row-echelon form using partial pivoting yields 


[аге] 


2 Rows 1 and 2 are interchanged 
1.000 because row 2 has the largest 
element in column 1, the current 
work column. 


1 
| 1 1 
— 0.00001 1 


1 ii 2 Rounding to four significant 
E 0 ii 1 figures. 
The system of equations associated with the last augmented matrix is consistent and is 
x,+x,=2 
x,=1 


Its solution is x, = x, = 1, which is also the solution to the original set of equations. 
All computers round to a number of significant figures k that depends on the machine being used. 
Then an equation of the form 
10 77x, +x, = 14104" 


will generate results like that of part a unless some pivoting strategy is used. (We had k = 4 in part a.) as 
a rule, dividing by very small numbers can lead to significant roundoff error and should be avoided when 
possible. 


2.10 Solve the following set of equations using partial pivoting: 


x, +2x,+ 3x,= 18 
2x, + x,- 4x,=~-30 
-5x,*8x,*17x,- 96 


The augmented matrix for this system is 
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In transforming this matrix, we need to use Step 1.3’ immediately, with R = 1 and C = 1. The largest 
element in absolute value in column 1 is —5, appearing in row 3. We interchange the first and third rows, 
and then continue the transformation to row-echelon form: 


1 -16 -341-192 
0 42 28| 84 
э[0 36 64! 37.2 


We next apply Step 1.3’ with R = 2 and C = 2. Considering only rows 2 and 3, we find that largest 
element in absolute value in column 2 is 4.2, so Z = 2 and no row interchange is required. Continuing 
with the Gaussian elimination, we calculate 


1-16  -34 $-19.2 
=o 1 0.666667: 2 

0 3.6 64 : 37.2 

Жы». -34 i-192 

0 34 0.666667: 2 
10 0 4 1 30 

1 -L6 -34 }-192 

0 1 0.666667 Н 2 
10 0 1 | 7.5 

The system of equations associated with the last augmented matrix is consistent and is 
x, ~ L6x, - 3.4x, = -192 
x, + 0.666667x, = 2 
xy 


Using back substitution (beginning with x,), we obtain, as the solution to this set of equations as well as 
the original system, x, = 1.5, x, = -3, and x, =7.5. 


2.11 To use scaled pivoting, we first define, as the scale factor for each row of the coefficient matrix 
A, the largest element in absolute value appearing in that row. The scale factors are computed 
once and only once and, for easy reference, are added onto the augmented matrix [A | B] as 
another partitioned column. Then Step 1.3 of Chapter 1 is replaced with the following: 

Divide the absolute value of each nonzero element that is in the work column and on or 
below the work row by the scale factor for its row. The element yielding the largest quotient is 
the new pivot; denote its row as row /. If row / is different from the current work row (row 
R), then interchange rows / and R. Row interchanges are the only elementary row operations 
that are performed on the scale factors; all other steps in the Gaussian elimination are limited 
to A and B. 

Solve Problem 2.10 using scaled pivoting. 


The scale factors for the system of Problem 2.10 are 
з, = max(1,2,3) =3 
5, = max(2,1,|-4]) =4 
s, = max{]-5], 8, 17} = 17 
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2.12 


We add a column consisting of these scale factors to the augmented matrix for the system, and then 
transforming it to row-echelon form as follows: 


12 зг 18713 The scale-factor quotients for the 
2 1-41-30 | 4| elements in column J are 1/3 = 
-5 8 17i 96 |17] 0.333, 2/4= 0.500, and 5/17 = 
0.294. 
=f 2 1 -41-30| 4 The largest quotient is 0.500, so 
t2 Зк 3&| 3 the pivot is 2, which appears in 
-5 8 17! 17] row 2. Since / - 2 and R - 1, the 
first and second rows are 
interchanged. 
>f 1 05 -21-15| 4 
1-2 3v 8] 3 
-5 8 17i 96117. 
1 05 -2i-15| 4 
>| 015 5! 33] 3 
1-5 8 17: 96117. 
1 0.5 -21-15 | 4] Мом work row is 2, and the work 
0 LS 5! 33 | 3| column is 2. The quotients are 
—lo 105 7: 21 [17] 1.5/3 =0.500 and 10.5/17 = 0.618. 
[1 05 -21-15 | 4 The largest quotient is 0.618, so 
0 105 7! 21 |17| the pivot is 10.5, which appears in 
—lo 15 5! 33 | 3]  row3. The second and third rows 
are interchanged. 
[1 05 rz 1-15] 4 
—|0 1 06667: 2]|17 
OS 55-1134 .3 
fros -2 1-5] # 
0 1 0.66667! 2/17 
lo 0 4 Жай 3 
105 -2 1-15 | 4 
E 1 0.66667; 2 |17 
Lo 0 1 i 2515.3 


Writing the set of equations associated with this augmented matrix (ignoring the column of scale factors) 
and solving them by back substitution, we obtain the solution x, = 1.5, x, = —3, х, = 7.5. 


To use complete pivoting, we replace Step 1.3 of Chapter 1 with the following steps, which 
involve both row and column interchanges: Let the current work row be R, and the current 
work column C. Scan all the elements of submatrix A of the augmented matrix that are on or 
below row R and on or to the right of column C, to determine which is largest in absolute 
value. Denote the row and column in which this element appears as row / and column J. If 
15% К, interchange rows / and R; if J C, interchange rows J and C. Because column 
interchanges change the order of the unknowns, a bookkeeping mechanism for associating 
columns with unknowns must be implemented. To do so, add a new partitioned row, row 0, 
above the usual agumented matrix. Its elements, which are initially in the order 1, 2, . . . , n to 
denote the subscripts on the unknowns, will designate which unknown is associated with each 
column. 
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Solve the system of Problem 2.10 using complete pivoting. We add the bookkeeping row 0 to the 
augmented matrix of Problem 2.10. Then, beginning with row 1, we transform the remaining rows into 
row-echelon form. 


R= 1 and C=1. The largest 
element in absolute value in the 
lower left submatrix is 17, in row 
3 and column 3. We first 
interchange rows | and 3, and 
then columns 1 and 3. 


1 0.470588 
0 2.88235 0.823528 
—L0 0.588236 1.88235 
3 2 The work row and work column 
_,| 1 0.470588 are now R =2 and С = 2. The 
0 1 largest element in absolute value 
10 0.588236 of the four under consideration is 
2.88235, for which / = 2 and J = 2. 
Since /= R and J 
interchange is гед 
E 2. - 
1 0.470588 Ё А 
0 1 0.285714 } —2.57143 
10 0 1.71428 $ 2.57143 
3 Sis 
T 1 8.64706 
7^ 1 0.285714 ; —2.57143 
0 0 1 i 1.50001 


The first column of the resulting row-echelon matrix corresponds to x,, and the third column to x,, so 


the associated set of equations is 


x, + 0.470588x, — 0.294118x, = 5.64706 
x, + 0.285714x, = -2.57143 
x,= 1.50001 


1 
Solving each equation for the first variable with a nonzero coefficient, we obtain 
X,= 5.64706 — 0.470588x, + 0.294118х, 
x, = 2.57143 — 0.285714x, 
1.50001 


which, when solved by back substitution. yields the solution x, = 1.50001, х, = —3.00000, and х, = 
7.50001. 


x= 
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Gauss-Jordan elimination adds a step between Steps 2.3 and 2.4 of the algorithm for Gaussian 
elimination. Once the augmented matrix has been reduced to row-echelon form, it is then 
reduced still further. Beginning with the last pivot element and continuing sequentially 
backward to the first, each pivot element is used to transform all other elements in its column 
to zero. 

Use Gauss-Jordan elimination to solve Problem 2.8. 

The first two steps of the Gaussian elimination algorithm are used to reduce the augmented matrix 
to row-echelon form as in Problems 1.13 and 2.8: 


T 2 
01 
00 


Then the matrix is reduced further, as follows: 


wh? 6 Add ~6 times the third row to the 
Ut 2 second row. 

1-1 
i | Add the third row to the first row. 
Ж: 
б©Є1 


— 
co- 
о-о cen cen 


0 
0 
1 
ot H Add —2 times the second row to 
0:2 the first row. 

1: 


=I 
The set of equations associated with this augmented matrix is x, = 1, x, = 2, and x, = —1, which is the 


solution set for the original system (no back substitution is required). 


Use Gauss-Jordan elimination to solve the system of Problem 2.7. 


The first two steps of the Gaussian elimination algorithm provide the augmented row-echelon-form 


matrix 
1 1/2 -1/210 
[C]D]-|O 1 91710 
0 0 о 10 


as in Problem 2.7. This matrix is reduced further by using the pivot in the (2,2) position to place а zero 
in the (1,2) position: 


[1 0 -8/7;0 
01 9/710 
оо о jū 
The set of equations associated with this augmented matrix is 


Add -1/2 times the second row to 
the first row. 


x,-$x,70 
x, + $x,=0 
0-0 


Solving for the first variable in each equation with a nonzero coefficient, we obtain x, = $x, and 
x, = -3x,, which is the solution (no back substitution is required) with x, aribitrary. 
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Supplementary Problems 


Which of 
(а) x, =x,=x,=1 (b) x, =8,x,=—1,x,=0 
(с) x, = 12, х= -3,х,=2 (4) x, = 2,2, = -2,x,-9 


are solutions to the system 
х.+3х+ x= 5 
2X, х,-Зху=1$ 
x,+7x,+5x,= 1 


Write the augmented matrix for the system given in Problem 2.15. 


Write the augmented matrix for system 


2x,-4x, t 7x, € Óx,—. 4х, = 17 
6x,-3x,- 4x,- 5x,= 2 
2x, +8x,+ x,-2x,—- 14x, = 10 


Solve the set of equations associated with each of the following augmented matrices: 


1 -2 3417 132 1:3 
(а) |» 1 245-3 (b) |» O 1 245 
4 


0 01: 00 0 1:0 
Solve the system given in Problem 2.15. 


Solve the system given in Problem 2.17. 


In Problems 2.21 through 2.27, solve for the unknowns in the given system. 


x,*2x,- ху=0 2.22 x,+2x,+3x,= 4 
2x, -2x, +3x,=0 4x, + 5x, 6x, = 16 
3x, x +2x,=0 Tx, + 8x, + 9x, =28 


2x,- x, +4x,= 6 22A х, +2х,+3х,+4х,= 8 
х; P3xy 43x92 —7 2x, -2x,— жү x,=-3 
х, + 2x, =-12 x, —3x,+4x,-4x,= 8 


2x, +2x, - 3x, + 4x, = -2 


bx, + Sx, + 4x, + $x, = 10 
tx, + bx, tix dx, = 


Ax, + dx, + $x, + $x, = 12 
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1.0001x, + 2.0000x, + 3.0000x, + 4.0000x, = 5 
1.0000х, + 2.0001х, + 3.0000х, + 4.0000x, = 6 
1.0000x, + 2.0000x, + 3.0001x, + 4.0000x, = 7 
1.0000х, + 2.0000x, + 3.0000х, + 4.0001x, =8 


What would be the result of solving this system by working with only four significant figures? 


0.00001x, + x, + 0.00001x, = 0.00002 
x *2x,* x,71 
0.00001х, + х, — 0.000015, = 0.00001 


Use Gaussian elimination to determine values of k for which solutions exist to the following systems, and 
then find the solutions: 


(а) x, +2x,- ху=4 (b) x,-3x,--4 
2х,- x, +3x,=3 2x,+ x, = 6 
3x,+ x, +2x,=k 3x,-2x,= k 


A manufacturer produces three types of desks: custom, deluxe, and regular. Each custom desk c 
requires 12 worker hours to cut and assemble, and 5 worker hours to finish. Each deluxe desk d requires 
10 hours to cut and assemble, and 3 hours to finish; each regular desk r requires 6 hours to cut and 
assemble, and 1 hour to finish. On a daily basis, the manufacturer has available 440 worker hours for 
cutting and assembling, and 120 worker hours for finishing. Show that the problem of determining how 
many desks of each type to produce so that all workpower is used is equivalent to solving two equations 
in the three unknowns c, d, and r. How many solutions are there? 


The end-of-the-year employee bonus b is 3 percent of taxable income i after city and state taxes are 
deducted. The city tax c is 1 percent of taxable income, while the state tax s is 4 percent of taxable 
income with credit allowed for the city tax as a pretax deduction. Show that the problem of determining 
the bonus is equivalent to solving three equations in the four unknowns Б, i, c, and s. 


Prove that if Y and Z are two solutions of the linear system AX = B, then Y — Z is a solution of the 
homogeneous system AX = 0. 


Prove that if Y and Z are two solutions of the linear system AX = B, then Y =Z +H, where Н is a 
solution of the homogeneous system AX = 0. 


Chapter 3 


Square Matrices 


DIAGONALS 


A matrix is square if it has the same number of rows and columns. Its general form is then 


4n di 8 oct dy 

а 05 05 ^05 

A-|44 а ау а, 

ал а, Any с Qnn 
The elements а, 4, A33, . .., 4„„ lie on and form the diagonal, also called the main diagonal or 
principal diagonal. The elements @,,, 2,3... . ,а, .,,,, immediately above the diagonal elements form 


the superdiagonal, and the elements a,,, a 
constitute the subdiagonal. 

A diagonal matrix is a square matrix in which all elements not on the main diagonal are equal to 
zero; the diagonal elements may have any values. An identity matrix 1 is a diagonal matrix in which 
all of the diagonal elements are equal to unity. The 2 x 2 and 4 x 4 identity matrices are 


4, „-ı immediately below the diagonal elements 


1000 
[ni «е foono 
0001 


Identity matrices play the same role in matrix arithmetic as the number 1 plays in real-number 
arithmetic. In particular, for any matrix А, Al = A and IA = A provided, in each case, that I is of the 
appropriate order for the indicated multiplication. 


ELEMENTARY MATRICES 


An elementary matrix E is a square matrix that generates an elementary row operation on a given 
matrix A under the multiplication EA. The order of E is dictated by the order of A, such that the 
multiplication is defined. There are three general kinds of elementary matrices, corresponding to the 
three different elementary row operations (see Chapter 1). A specific elementary matrix is obtained 
by applying the desired elementary row operation to an identity matrix of the appropriate order. 
(See Problems 3.1 and 3.2.) 


LU DECOMPOSITION 


A square matrix is upper triangular if all elements below the main diagonal are zero; it is lower 
triangular if all elements above the main diagonal are zero. The elements on or above the diagonal in 
an upper triangular matrix (and on or below the diagonal in a lower triangular matrix) may have any 
values, including zero. 

In most cases, a square matrix A can be written as the product of a lower triangular matrix L and 
an upper triangular matrix U, where L and U have the same order as A. This factorization, when it 
exists. is unique if the elements on the main diagonal of U are all 1s. That is. 


A=LU (3.1) 
24 
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1, 0 0 0 
n b 0 0 
where ъ= М | 9 and 
[m ban 
2 L Е 2 о 0 1 1/2 1/2 
Ехатріе 3.1 2 =] rpe€2 c 0 0 1 0 
4 1. 1 4 -1-1 0 0 1 


Crout’s reduction is an algorithm for calculating the elements of L and U. In this procedure, the 
first column of L is determined first, then the first row of U, the second column of L, the second row 
of U, the third column of L, the third row of U, and so on until all elements have been found. The 
order of L and U is the same as that of A, which we here assume is n X n. 


STEP 3.1: Initialization: If a,, = 0, stop; factorization is not possible. Otherwise, the first column 
of L is the first column of A; remaining elements of the first row of L are zero. The first 
row of U is the first row of A divided by /,, = a,,; remaining elements of the first column 
of U are zero. Set a counter at N = 2. 

STEP 3.2: For i=N,N+1,...,m, set L; equal to that portion of the ith row of L that has 
already been determined. That is, L; consists of the first N — 1 elements of the ith row 
of L. 

STEP 3.3: For j=N,N+1,...,m, set U; equal to that portion of the jth column of U that has 
already been determined. That is, U; consists of the first N — 1 elements of the jth 
column of U. 

STEP 3.4: Compute the Nth column of L. For each element of that column on or below the main 
diagonal, compute 


liy=@y—-(Li)"*Uy (= N,N 41,...,0) 


If any ly, =0 when № # n, stop; the factorization is not possible. Otherwise, set the 
remaining elements of the Nth row of L equal to zero. 

STEP 3.5: Set uyy = 1. If N = n, stop; the factorization is complete. Otherwise, set the remaining 
elements of the Nth column of U equal to zero and compute the Nth row of U. For each 
element of that row to the right of the main diagonal, compute 


ам 7 (L3) U; 


Uy, = 
ii lyy 


] 


(J=N41,N42,...,n) 


STEP 3.6: Increase N by 1, and return to Step 3.2. 


(See Problems 3.4 through 3.6.) 

Partial pivoting (see Chapter 2) is recommended when exact arithmetic is not used and roundoff 
error is possible. Prior to Steps 3.1 and 3.2 (for N = 2,3,..., п), scan the Nth column of A for the 
largest element in absolute value appearing in that column and on or below the main diagonal. If this 
element is in row p, with p # №, then interchange the pth and Nth rows of A, as well as the pth and 
Nth rows L up to the Nth column (which represents the parts of those two rows in L that have 
already been determined). 


SIMULTANEOUS LINEAR EQUATIONS 


LU decompositions are useful for solving systems of simultaneous linear equations when the 
number of unknowns is equal to the number of equations. The matrix form of such a system is 
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AX=B, which, in light of Eq. (3.1), may be rewritten as L(UX)=B. To obtain X, we first 
decompose A and then solve the system associated with 


LY=B (3.2) 
for Y. Then, once Y is known, we solve the system associated with 
UX-Y (3.3) 


for X. Both (3.2) and (3.3) are easy to solve—the first by forward substitution, and the second by 
backward substitution. (See Problem 3.7.) 

When A is a square matrix, LU factorization and Gaussian elimination are equally efficient for 
solving a single set of equations. LU factorization is superior when the system AX = B must be solved 
repeatedly with different right sides, because the same LU factorization of A is used for all B. (See 
Problem 3.8.) A drawback with LU factorization is that the factorization does not exist when a pivot 
element is zero. However, this rarely occurs in practice, and the problem can usually be eliminated 
by reordering the equations. Gaussian elimination is applicable to all systems, and for that reason is 
often the preferred algorithm. 


POWERS OF A MATRIX 
If n is a positive integer and A is a square matrix, then 
А" = ДА.А 
n times 
In particular, А? = AA and А? = AAA. By definition A" =I. (See Problems 3.10 and 3.11.) 


Solved Problems 


3., Find elementary matrices that when multiplied on the right by any 3 x 3 matrix A will (a) 
interchange the first and third rows of A; (b) multiply the second row of A by 1/2; and (c) add 
—4 times the second row of A to the third row of A. 


Since an elementary matrix is constructed by performing the desired elementary row operation on 
an identity matrix of the appropriate size, in this case the 3 x 3 identity, we have 


001 1 0 0 1 00 
(а) E=/0 1 0 (b) Е=|0 12 0 (с) E=/0 10 
100 0 0 1 0 -4 1 
3.2 Find elementary matrices that when multiplied on the right by any 4 x 4 matrix A will (a) 


interchange the second and fourth rows of A; (Б) multiply the third row of A by —6; and (c) 
add 8 times the first row of A to the fourth row of A. 


1000 10 00 1000 
0001 от оо отоо 
@ Е=о o 1 of © Е=јоо -6 o| © F=|0 010 
0100 00 01 8001 
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Find a matrix P such that PA is in row-echelon form when 


EboX ek 
A-|3 8 9 
2 b g 


The matrix A consists of the first three columns of the matrix considered in Problem 1.13, so the 
same sequence of elementary row operations utilized in that problem will convert this matrix to 
row-echelon form. The elementary matrices corresponding to those operations are, sequentially, 


100 1.0 0 
E-|-310 0 1/2 0 
001 


1 0 0 
P=E,E,E,E,E,=| -3/2 12 0 


Then 
—19/68 5/68 2/68 
1 0 0 1 d od t 2) 
and РА =] -3/2 1/2 0 |3 8 9[=|01 6 
-19/68 5/68 2/6812 -1 2 00 1 
Factor the following matrix into an upper triangular matrix and a lower triangular matrix: 
Lr 2-2 3 
xp elc de © E 
A=] 3 -3 4 1 
2 1 1-3 
Using Crout's reduction, we have 
STEP 3.1: 
00 12-2 3 
as | 2M ime sm woe = - 
в=| у 17] Ы 77) N=2 
EEE" 0- =- 
STEP 32: L;=[-1), L; = [3), and L; 
STEP 3.3: U; = [2], О; = [-2], and U; = [3] 
STEP 3.4: 
1745 - ()'-U; = 1 - [-1]*[2} = 1- (-2)=3 
1» = ay - (L5) +U; = -3- [3]: [2}= -3-6 = -9 
hag = а. ~ (Li) *U; 21- [2] [2]= 1-4 = -3 
STEP 3.5: 
и =1 
ata — (U3) _0-[-1]{-2]__2 
„= [TR 3 "S 


-(L)-U; 
n Ds 3 3 
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STEP 3.6: To this point we have 


1 000 i3 =H. 5 
-1 300 [0 1 -25 5/3 
L= 3-9-- and U- QU - 5 
2 34 Su m x 
Since N =2 and n = 4, we increase N by 1 to N = 3. 
STEP 32: L; =[3, —9] and L; =[2, -3]. 
STEP 3.3: 
Ec "M ТҮЗ 
u-[X] s vfi] 
STEP 34: a 
y= ay 7 (4) 05247 [ 2] [ 25] 4-0-4 
WEE 2). -2 
day = ay ~ 02"; 21-[ 2 [5]21- 025 
STEP 3.5: 
us] 
ам M, [ 3L 3 1-09. 7 
wr lj -40 [5] BA) 4 4 
STEP 3.6: To this point we have 
1 000 012-2 3 
da зоо [о 1 -23 5/3 
b*| 3294 0) 8 Шоо 4 тщ 
2-33 = 00 0 - 


Since М =3 and n = 4, we increase N by 1 to № = 4. 


STEP 3.2: 14 =3;3]; 
3 
U, =|5/3 
7/4 


STEP 3.3: 
2773 
мада а-13 s] - 2 = E 
3 


STEP 3.4: 


714 
STEP 3.5: и, 7 1. Since N = 4 = n, the factorization is done. We have А = LU, with 


1 00 0 їй 2 8 

-1 30 0 01 -23 5/3 
Le} $9.4 0 a. US. Үү 4 
2 -3 3 -33/4 00 0 1 


Т. 3 6 
eic» 9: 3 
AS 3-3: di d 

2 1 1-2 


The first three steps of Grout's reduction here are identical to those in Problem 3.4. Then: 


STEP 3.4: 
= а - 24) +U; = -2- [-1]:02] = -2- (-2)=0 


Since /„, = 0 but № # п, the original matrix cannot be factored as an LU product. 
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3.6 Factor the following matrix into an upper triangular matrix and a lower triangular matrix: 


22 r0 
30-11 
aml ox 9:5 
11 090 
Using Crout's reduction, we have 
STEP 3.1: 
2000 1 1/2 0 
43--- "n pure» " 
L= icu ө U- ж.ж М=2 
Ea Жы Oi: aS 


STEP 3.2: V; = [3], L; = [0], and L; = [-1]. 

STEP 3.3: Us = [1], U; = [1/2], and U; = [0]. 

STEP 3.4: 15745 7 (L3)” + Us 20- [3] -[1] = 0- (3)  -3 
ly = а ~ (L5)7+U, = 1- [Q] [1] 21-071 
fag 744 7 (1) U; = 1- [1] (1 2 1- (71) 22 


STEP 3.5: 
из =1 
„ЮЗ; 1-10] _ 5 
a ts -3 6 
u o a (lU i-o). d 
a L, =3 
STEP 3.6: To this point we have 
2 000 i112 0 
$3 0 0 210 1 5/6 -1/3 
bea a = a) os ag OE 
=} 25 = бй x a 


Since N —2 and n = 4, we increase N by 1 to № = 3. 
STEP 3.2: L; = [0,1] and L; = [-1,2]. 


STEP 3.3: 
,.[U2 — 
%=[5% тыч vef ia] 
STEP 3.4: 
Sape о] [12 5 5 
ота 12 1 
но HE i 0-35-53 
STEP 3.5: 


(s- -[ n Vis 516) = 52 CUS S13. 2 


30 
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STEP 3.6: To this point we have 


2 оо 0 1112 0 
_| 3 +3 0 0 01 5/6 -1/3 
b=) o 1 -s6 0| 21 U-loo 1 -xs 

- 3 ue = фый 0 s 


Since № =3 and n = 4, we increase № by 1 to № = 4. 
STEP 32: L,-|-1,2, 7/6]. 
STEP 3.3: 


STEP 3.4: 


E D » 
hazau- aprum] 2 H ap te-- n 


-7/6 -32/5 


STEP 3.5: u,,=1. Since N=4=n, the factorization is done. We have A= LU, with 


20 0 0 y 5 
3-30 0 АГЕ 
L^5 ò 1 -s6 0 ad U= 
-1 2 -T/6 -34/5 00 


Solve the system of equations 


2x, + 2x, +25 = 10 
3x, -x,* x,=-11 

x, +5x,= 5 
=x, + X, = 14 


1/2 0 
5/6 -1/3 
1 -32/5 
0 1 
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The coefficient matrix for this system is matrix A of Problem 3.6. Using L as determined in that 


problem, we write the system corresponding to LY = B as 


2y, = 10 
Зу, —3y2 =-11 
yi-5/6y, = 5 


=y, *2y,-7/6y,- 34/5у,= 14 


Solving this system sequentially from top to bottom, we obtain y, = 5, у, = 26/3, y, = 22/5, and 


у= -1. 


With these values апа U as given in Problem 3.6, we can write (ће system corresponding to UX = Y: 


x, txt bn 
ху; 


Solving this system sequentially from bottom to top, we obtain the solution to the original system: 


x, = —4, x, = 10, x, = —2, and x, = - 1. 
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3.8 — Solve the system of equations given in Problem 3.7 if the right side of the second equation is 


changed from —11 to 11. 


The coefficient matrix A is unchanged, so both L and U are as they were. From (3.2), 


2y, =10 
3y, -3y. = 
Yam bys = 5 


-y, +2у,- dy, - у, = 14 


Solving this system sequentially from top to bottom, we obtain y, =5, y, = 4/3, y, = -22/5, and 


y, = 28/17. With these values and U as given in Problem 3.6, (3.3) becomes 


xt + ix = 


Solving this system sequentially from bottom to top, we obtain the solution to the system of interest: 


x, = -13/17, x, = 225/17, x, = —254/17, and x, = —28/17. 


3.9 Verify Crout's algorithm for 3 x 3 matrices. 


For an arbitrary 3 x 3 matrix A, we seek a factorization of the form 


ау а as] [h 0 OFT ua s] [in hs ии, 
аһ аһ Ary f=} lay ba OfO 1 и, |= |, inutii шу + beats 


ау а d» э bh 630 0 1 ly bata tha шз + betas s 


By equating corresponding coefficients in the order of first column, remaining first row, remaining 
second column, remaining second row, and remaining third column, and then solving successively for the 


single unknown in each equation, we would obtain the formulas of the Crout reduction algorithm. 
3.10 Find А? and A‘ when 
А a] 
ve [SB uoo 3] 
docu Er EH 25] 


3.14 Show that А? — 9A + 101= 0 when 


We have 


and 


32 


3.12 


3.15 


3.16 


3.17 
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-1 -8 18 1-72 2 100 000 
A'-9A«101-| 0 8 01-90 2 0/+10]0 1 0j=j0 0 0 
9. —9:—3 Б =] =3; 001 00 0. 


A square matrix A is said to be nilpotent if А? = 0 for some positive integer р. If p is the least 
positive integer for which А? = 0, then A is said to be nilpotent of index p. Show that 


is nilpotent of index 3. 
That is indeed the case, because 


15 -2pi 5 2] fo 3 -1 
A=}1 2 -1J|[1 2 -1]=/0 3 -1 
3 6 -3]3 6 -3] lo 9 -3 


and 03-1171 5 -2] fo 0 0 
A-AA-|0 3 -1|1 2 -1]=10 0 0 
о 9 -3]3 6 -3] Lo o o 


Supplementary Problems 


Find elementary matrices that, when multiplied on the right by any 3 x 3 matrix A (a) will interchange 
the second and third rows of A; (b) will multiply the first row of A by 7; and (c) will add —3 times the 
first row of A to the second row of A. 


Find elementary matrices that when multiplied on the right by any 4 x 4 matrix A (a) will interchange 
the second and third rows of A; (5) will add —3 times the first row of A to the fourth row of A; and (c) 
will add 5 times the third row of A to the first row of A. 


Find (a) a matrix P such that PA is in row-echelon form and (b) a matrix Q such that QA =I when 


^i 


Use elementary matrices to find a matrix P such that PA = 1 when 


102 
A=/0 12 
3 2.5 


Prove that the product of two lower triangular matrices of the same order is itself lower triangular. 


In Problems 3.18 through 3.23, write each of the given matrices as the product of a lower triangular matrix 
and an upper triangular matrix. 


3.18 


1 23 3.19 2 эй 3.20 
456 133 
78 9 =l 2 0 


CONS 
wneo 
onoo 
-omn 
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3.21 


3.23 


3.25 


3.26 


327 


3.28 


3.30 


3.31 


3.32 


3.33 


3.34 


l 2 3 4 322 [1 з 3 4 
2 4$ -L i A x2 pL oH 
à «3 £4 L= 4 oc 
2 2-3 4 2. 23" 4 


In Problems 3.23 through 3.29, use LU factorization to solve for the unknowns. 


2x, t 2x, xs - 3.24 x +2x,-2x,+3x,= 2 
3x, -x*on- 
s +5x,= 
-x+ x, = 
(Hint: See Problems 3.7 and 3.8.) (Hint: See Problem 3.4.) 


Repeat Problem 3.24, but with B =[-3, —1, 0, 4,]". 


x, +2x,+3x,=4 
4x, + Sx, + 6x, = 16 
7x, + 8x, + 9x, = 28 
(Hint: See Problem 3.18.) 


Repeat Problem 3.26, but with B = [6, —7, 12". 


2x,- x, +4x,= 6 3.29 x, +2x,+3x, +4x,= 8 
x, +3x,+3x,= -7 2x, —2x,- x,* x,=-3 
—x,+2x, =-12 x, —3x, + 4x, — 4x, 8 
(Hint: See Problem 3.19.) 2x, 82x, = 3x, + 4x, = -2 


(Hint: See Problem 3.21.) 
Find A’ and A’ for the matrix given in Problem 3.15. 
Find A* for 
20 0 
A=|0 1 0 
00 -1 
What does A^ look like when A is a diagonal matrix? 


A square matrix is said to be idempotent if А? = A. Show that the following matrix is idempotent: 


22 =a 
A=/|-1 3 4 
1 2 3 


Prove that if A is idempotent, then so too is I — A. 


Prove that (A^) = (А”)”. 
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Matrix Inversion 


THE INVERSE 
Matrix B is the inverse of a square matrix A if 
AB-BA-I (4.1) 


For both products to be defined simultaneously, A and B must be square matrices of the same order. 
Example 4.1 
[52 4] isthe inverse of [3 3] 
1 2 1 1р2 1 1 2],[1 0 
решш L 4052 -l-l -ias 41-16 1] 


A square matrix is said to be singular if it does not have ап inverse; a matrix that has an inverse 
is called nonsingular or invertible. The inverse of A, when it exists, is denoted as A'*. 


SIMPLE INVERSES 


Elementary matrices corresponding to elementary row operations (see Chapter 3) are invertible. 
An elementary matrix of the first kind, one that corresponds to an interchange of two rows, is its own 
inverse. The inverse of an elementary matrix of the second kind, one that corresponds to multiplying 
one row of a matrix by a nonzero scalar k, is obtained simply by replacing the value of k in the 
elementary matrix with its reciprocal 1/k. The inverse of an elementary matrix of the third kind, 
which corresponds to adding to one row a constant k times another row, is obtained by replacing the 
value k in the elementary matrix with its additive inverse —k. (See Problem 4.2.) 

The inverse of an upper triangular matrix is itself upper triangular, while that of a lower 
triangular matrix is lower triangular (see Problem 4.13), provided none of the diagonal elements is 
zero. If at least one diagonal element is zero, then no inverse exists. The inverses of triangular 
matrices are constructed iteratively, one column at a time, using Eq. (4.1). (See Problems 4.3 and 
4.4.) 


CALCULATING INVERSES 


Inverses may be found through the use of elementary row operations (see Chapter 1). This 
procedure not only yields the inverse when it exists, but also indicates when the inverse does not 
exist. An algorithm for finding the inverse of a matrix A is as follows: 


STEP 4.1: Form the partitioned matrix [A | 1], where Lis the identity matrix having the same order 
as A. 

STEP 4.2: Using elementary row operations, transform A into row-echelon form (see Chapter 1), 
applying each row operation to the entire partitioned matrix formed in Step 1. Denote 
the result as [С | D], where C is in row-echelon form. 

STEP 4.3: 1f C has a zero row, stop; the original matrix A is singular and does not have an inverse. 
Otherwise continue; the original matrix is invertible. 
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STEP 4.4: Beginning with the last column of C and progressing backward iteratively through the 
second column, use elementary row operation E3 to transform all elements above the 
diagonal of C to zero. Apply each operation, however, to the entire matrix [C | D]. 
Denote the result as [I| B]. The matrix B is the inverse of the original matrix A. 


(See Problems 4.5 through 4.7.) If exact arithmetic is not used in Step 4.2, then a pivoting strategy 
(see Chapter 2) should be employed. No pivoting strategy is used in Step 4.4; the pivot is always one 
of the unity elements on the diagonal of C. Interchanging any rows after Step 4.2 has been completed 
will undo the work of that step and, therefore, is not allowed. 


SIMULTANEOUS LINEAR EQUATIONS 
A set of linear equations in the matrix form 
AX=B (4.2) 


can be solved easily if A is invertible and its inverse is known. Multiplying each side of this matrix 
equation by А”! yields A 'AX = A'!B, which simplifies to 


X-A'B (4.3) 


(See Problems 4.8 and 4.9.) Equation (4.3) is most useful as a theoretical representation of the 
solution to (4.2). The methods given in Chapter 2 for solving simultaneous linear equations generally 
require fewer computations than the method indicated in (4.3) when A”! is not known. 


PROPERTIES OF THE INVERSE 

Property 4.1: The inverse of a nonsingular matrix is unique. 

Property 4.2: If A is nonsingular, then (A^!) ! =A. 

Property 4.3: If A and B are nonsingular, then (AB)! = B "A''. 

Property 4.4: If A is nonsingular, then so too is A. Further, (А?) | = (A !)". 
(See Problems 4.10 to 4.12 and 4.30.) 


Solved Problems 


41 Determine whether 


6=[ os оз 


is the inverse of any of the following matrices: 


_ К 2-4 0 
eun ей i: 


We consider each of the given matrices in turn. Since 


лв=[4 oll- 05-05 2] 


is not the identity matrix, G is not the inverse of А. 
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B is not square, so it has no inverse. In particular, the product BG is not defined. 
For C, matrix multiplication gives 


co-f pE- ЫН d 


and сс-[_0,; om [; ~ol- [o | 


so G is the inverse of C. 
G and D do not have the same order, so they cannot be inverses of one another. 


4.2 Determine the inverses of the following elementary matrices: 


001 100 400 
А=|0 1 0 B=|0 0 | С=|0 1 0 
100 010 001 
0 0 

1 0 

1 


1 00 1 0 10 
D-|0 2 0 E=|0 0 Е=|-3 1 
0 01 0:52: 1 00 


Both A and B are elementary matrices of the first kind; thus, A^' = A and B ' = B. Matrices C and 
D are elementary matrices of the second kind. Their inverses are 


иа 0 0 1 о 0 
с'=| 0 10 and D'^'-j|0 -1/2 0 
0 01 0 0 1 
Matrices E and F are elementary matrices of the third kind. Their inverses are 


1 00 100 
E'=|0 10 and =F '=/3 1 0 


0-21 001 


43 Determine the inverse of 


Since A is upper triangular with no zero elements on its diagonal, it has an inverse and the inverse is 
upper triangular. Furthermore, since А 'А = I, we may write 


ab сү2 1 3 100 
0 аео 1 2)=/0 10 
оо 10 0 3 001 


with the first matrix on the left representing A`'. We perform the indicated matrix multiplication and 
equate corresponding elements on and above the diagonal. Beginning with the leftmost column and 
sequentially moving through successive columns, we determine that 


a(2) + 600) + c(0) 2 1 
$(1) + 5(1) + c(0) =0 
O(1) + d(1) + e(0) =1 
1(3) + (— 2)0) + (3) =0 
0(3) + 1(2) + e(3) =0 
0(3) + 0(2) + f3) =1 


5556586868 
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M2 -1/2 -1/6 
Thus, A'-[0 p 273 
0 0 13 


4.4 Determine the inverses of 


з о 00 -1 0 00 
Ji - o (12-2 00 
^s». 4 тр 3-9 BS x f 20 

1 3-10 1 cb 83 


Both matrices are lower triangular. Since A has a zero element on its main diagonal, it does not 
have an inverse. In contrast, all the elements on the main diagonal of B are nonzero so it has an inverse 
which itself must be lower triangular. Since BB ' =I, we may write 


-1 0 00faooo]pooo 
2-2 0 off6 co o|_ отоо 
3 1 -2 Olid e f о| ооо 
1-1 3 31е h i jl looot 


with the second matrix on the left representing B ^'. We perform the indicated matrix multiplication and 
equate corresponding elements on and below the diagonal. Beginning with the leftmost column and 
sequentially moving through successive columns, we determine that 


—la+0b+0d+0g=1 so 
2(-1) + (-2)5 + 0d + 0g - 0 so 
3(-1)*1-71) *(-2)4*0g-0 зо 
\(-1) + (-1)(-1) + 3(-2) +3g=0 зо 
2(0) + (-2)c + 0e + 0h = 1 so 

3(0) + 1(-1/2) + (-2)e + 0h- 0 so 
1(0)*(-1(-1/2)*3(-1/4) +3h=0 — so 
3(0) + 100) + (-2)f + 0i so 

1(0) + (-1)(0) + 371/2) +3 so 
100) + (-1)(0) + 3(0) + 3/ = 1 so 

0 

0 


-1 0 0 
= -1 -1/2 0 
Therefore BY "l-2 -i4 -12 0 


2 ИП 1/2 n3 


45 Determine the inverse of 


We follow Steps 4.1 through 4.4, beginning with [A | I]: 


É 3:1 ч 
2 1:0 1 
> Е 0.6 0.2 m Multiplying the first row by 1/5 
АЖ c0 1 

Е 06: 02 0 Adding —2 times the first row to 
910 -02:-04 1 the second row 

5 0.6: 0.2 ?] Multiplying the second row by 
alo 1:12 -s5 -1/02 


38 MATRIX INVERSION CHAP. 4] 


The left side of this partitioned matrix is in row-echelon form. Since it contains no zero rows, the original 
matrix has an inverse. Applying Step 4.4 to the second column, we obtain 


> E 0:-1 2) Adding —0.6 times the second 


Of 32 =5 Tow to the first row 
afi ЧІ 
Therefore, AU= [ 2 -5 


4.6 Determine the inverse of 


12 3:100 
[Al=]4 5 6:0 1 0 
789:00 1 
ї 2 By 0:0 Adding —4 times the first row to 
710 =з -6:-4 10 the second row 
7 8 9:001 
ї 2 1100 Adding —7 times the first row to 
9:3 i410 the third row 
10 -6 zu X 
T- 2 "NP 0 0 Multiplying the second row by 
—-|0 1 2:4/3 -1/3 0 -43 
0.6 121-7. ШП | 


01 2:4/3 -13 0 the third row 


[ 23r" 0 o] Adding 6 times the second row to 
—251000:1 ”Ж. 


The left side of this partitioned matrix is in row-echelon form. Since its third row is zero, the original 
matrix does not have an inverse. 


4.7 Determine the inverse of 


=“ “Жой 0 1 
M5  -15;0 15 e] Adding —2 times the first row to 


5:0 1/5 o] Multiplying the first row by 1/5 


1 P :1 g Q0 
-17/5 -13/510 -2/5 1 


the third row 


0 1 1:100 
[А|]=|5 1 -1'010 
2: -3 310,0 1 
—-[5 1 -1!0 10 Interchanging the first and second 
—|0 1 1110 0| row 
2-3 -310 01 
Shr Ws = 
of Lost 10: <0 
2 
1 
0 
0 
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0 1 L4 0 0 
0 0 4/5: 17/5 -2/5 1 


to the third row 


1 05 -1/5; 0 15 0 Adding 17/5 times the second row 
ў 


o i Lfd 0 0 
0 0 1 :174 -2/4 5/4 


11/5 -15; 0 15 0 Multiplying the third row by 5/4 
- 
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The left side of this partitioned matrix is in row-echelon form and contains no zero rows; thus, the 


original matrix has an inverse. Applying Step 4.4, we obtain 


0 ! 0 :-13/4 2/4 -S/4 the second row 


0 0 1 : 17/4 -2/4 514 


[e 15 -15:; 0 ys 0 | Adding – 1 times the third row to 
ni 


-13/4 2/4 -S/4 
17/4 -2/4 5/4. 


the first row 


—f1 00; 64 0 2/4] Adding —1/5 times the second row 
| 10;-13/4 2/4 i] to the first row 
00 1; 17/4 -2/4 5/4 
6/4 0 24] sf 6 о 
Thus A"-|-134 2/4 ЕЕ 2 
17/4 -2/4 5/4 17 -2 


4.8 Solve the system 


5x, + 3х, =8 
2x, + x,2-1 


This system can be written in the matrix form 
5 3I [5]. [ 8 
[; iulz]- [s ] 
Using the result of Problem 4.5 with Eq. (4.3), we have 
EJ 3080-78) 
х, 2 511-1 21 
The solution is x, = —11, х= 21. 
4.9 Solve the system 
X,* ху=2 
Sx, t x- ху=3 
2x, — 3х, — 3x, = -6 
This system can he written in the matrix form 
0 1 fi 2 
5 1 jajj 3 
2 -3 -3]*5 -6 


Using the result of Problem 4.7 in Eq. (4.3), we have 


2 
-5 
5 


at 1/5 0; 17/20 1/10 s Adding 1/5 times the third row to 
0; 
1: 


| 


4.10 


4.11 


4.12 


4.13 


4.14 
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“| yf 6 0 2797 2 0 

ajag 2 -s| 3|-.|s2 

ху 17 -2 51-6] |-12 
The solution is x, = 0, x, 75/2, x, = —1/2. 


Prove that the inverse is unique when it exists. 
Assume that A has two inverses, В and C. Then АВ = І and CA = I. It follows that 


С= СІ = C(AB) = (СА)В = IB = B 


Prove that (A!) = A when A is nonsingular. 


(A ')^' is, by definition, the inverse of A`’. A also is the inverse of A`’, These inverses must be 
equal as a consequence of Problem 4.10. 


Prove that (AB)! = ВА! if both A and B are invertible. 
(АВ) `' is, by definition, the inverse of AB. Furthermore, 


(ВТА (AB) = B'(A"'AJB-B^'IB-B^'B-I 
and (ABY(B^'A7!) = A(BB ')А' A" = АА”! =1 


so В 'A ‘is also an inverse of AB. These inverses must be equal as a consequence of Problem 4.10. 


Prove that the inverse of a lower triangular matrix A with nonzero diagonal elements is itself 
lower triangular. 

The proof is inductive on the rows of A~". Denote the inverse of A = [а] as А”! = [а, |. Since the 
product AA" ' is the identity matrix, the element in the ith row and jth column of this product must be 
zero when i # j. In particular, the element in the first row and jth column of АА” !, with j > 1, is zero. We 
may write that element as 


07 X aua, 7 aua, + È ама, =аца, + У (Olay) = ana, 


kzi 


We are given a,, #0, which implies that а, =0 for j >1. 
Now assume that а, = 0 for j >i and all і = p — 1; compute the pth row of AA“. Since АА ' is the 
identity matrix, the element in its pth row and jth column, for j > р, must satisfy 
А p-1 А 
0= Ў аа, = У аа, +4,,0,,+ У a, 
En m m 
p ^ 
= 2 naNO аа, + У (Oa) =а,а,, 


M 


Since а, #0, it follows that a,, =0 when j>p. 


Prove that any square matrix that can be reduced to row-echelon form without rearranging 
any rows can be factored into a lower triangular matrix L times an upper triangular matrix U. 


The reduction of a matrix A to row-echelon form can be expressed as the product of a sequence of 
elementary matrices, one for each elementary row operation in the reduction process, multiplied by A. If 
U is the resulting row-echelon form of A, then U is upper triangular and 


(E,E,., E,E,)A =U a) 
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Each E, is an elementary matrix of either the second or third kind, so each is lower triangular and 
invertible. It follows from Problem 4.12 that if 

P-EE,. 
then P^ (EE, , EE 
Р”! is thus the product of lower triangular matrices and is itself lower triangular (Problem 3.17). From 
(1), PA = U whereupon 


(2) 


A-IA-(P^'PJA-P'(PA) = P'U 


Supplementary Problems 


In Problems 4.15 through 4.26 find the inverse of the given matrix if it exists. 


4.15 0001 1000 1000 100 0 
0100 0700 0100 010 -3 
€ oio| © fooro) © озто| © јоот o 
1000 0001 0001 000 1 
416 [20] “7 [э] «s [2 3] 
11 > ЖЕ. 4 6 
«s [3 4 ало [100 4n [1-12 
21 220 0-13 
поз 0 05 
42 [300] 423 [2 1 3] 4м [1-23 
-120 4 безе 3 51 
rig 2-1 1 6 42 
425 [1 21) 4% [22 3 3 
i ib-1 2332 
3 -1 $3 7 8 
L3 2 4 7 


In Problems 4.27 through 4.29, use matrix inversion to solve for the unknowns: 


4.27 2x, * x, 4+3x,=-3 428 x, +2x,+x,=0 
4x, *2x,—- ху= 5 x +x +x, =0 
2x,- х,+ х= 2 3x,-x,+x, =6 
(Hint: See Problem 4.23.) (Hint: See Problem 4.25.) 


429 2x, +2x, ^ 3x, «3x, 71 
2x, + 3x, + 3х, 42x, 71 
Sx, + 3х, + 71, 4 91, 1 
3x, 2x, áx, 473,71 
(Hint: See Problem 4.26.) 


4.30 Prove that (A ')' = (A). 


4.31 Prove that if the commutative property of multiplication holds for nonsingular matrices A and B, then it 
also holds for the following pairs of matrices: (а) A^ ' and B '; (b) A^' and B; (c) A and B^. 


Chapter 5 


Determinants 


EXPANSION BY COFACTORS 


The determinant of a square matrix A, denoted det A or |А|, is a scalar. If the matrix is written 
out as an array of elements, then its determinant is indicated by replacing the brackets with vertical 
lines. For 1 X 1 matrices, 


det A = |a |= à. 
For 2 x 2 matrices, 


а аә 
ап 02 


detA= = 44,05 аа 


Determinants for n x n matrices with n>2 are calculated through a process of reduction and 
expansion utilizing minors and cofactors, as follows. 


A minor M, of an n x n matrix A is the determinant of ће (n — 1) x (n — 1) submatrix that 
remains after the entire ith row and jth column have been deleted from A. 


Example 5.1 For 
012 
A=|3 4 5 
678 
45 
M, |5 6—48) sme -3 
м. |6 }|=0(7)- 106) = -6 
M, =|) 2-19 -20)= 3 


A cofactor A,, of an n X п matrix A is defined іп terms of its associated minor as 


A,7 (71M, 
Now for any i or j (i, /21,2,..., n), 
det A= У a4A4 7 У аА, (5.1) 
E ^ 


For each i, the first sum in (5.1) represents an expansion along the ith row of A; for each j, the 
second sum represents an expansion along the jth column of A. Choosing to expand along a row or 
column having many zeros, if it exists, greatly reduces the number of calculations required to 
compute det A. (See Problems 5.2 through 5.4.) 


PROPERTIES OF DETERMINANTS 


Property 5.1: If A and B are square matrices of the same order, then det AB = det A det B. 


Property 5.2: Тһе determinant of an upper or lower triangular square matrix is the product of the 
diagonal elements. 


42 
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Property 5.3: If B is formed from a square matrix А by interchanging two rows or two columns of 
A, then det А = -det B. 


Property 5.4: If B is formed from a square matrix A by multiplying every element of a row or 
column of A by a scalar k, then det А = (1/k) det В. 


Property 5.5: If B is formed from a square matrix A by adding a constant times one row (or 
column) of A to another row (or column) of A, then det A = det B. 


Property 5.6: If one row or one column of a square matrix is zero, its determinant is zero. 
Property 5.7: det A’ = det A, provided A is a square matrix. 
Property 5.8: If two rows of a square matrix are equal, its determinant is zero. 


Property 5.9: А matrix A (not necessarily square) has rank k if and only if it possesses at least one 
k x k submatrix with a nonzero determinant while all square submatrices of larger 
order have zero determinants. 


Property 5.10: If A has an inverse, then det A^! = 1/det A. 


DETERMINANTS OF PARTITIONED MATRICES 


A block matrix is one whose elements are themselves matrices. Property 5.2 can be extended to 
partitioned matrices in block upper (or lower) triangular form. If 


Ay Ai c Ay 
0 An © Ay 
ыы о о .- As, 
0 0 w A 


where each of the submatrices A,,, А,,,...,А,, is square, then 


det A = det A,, det A,, det A; -+ det A,, (5.2) 
(See Problem 5.8.) 


PIVOTAL CONDENSATION 


Properties 5.3 through 5.5 describe the effects of elementary row and column operations on a 
determinant. Combined with Property 5.2, they form the basis for the pivotal condensation algorithm 
for calculating the determinant of a matrix A, as follows: 


STEP 5.1: Initialize D — 1. D is a scalar that will record changes in det A as a result of elementary 
row operations. 

STEP 5.2: Use elementary row operations to reduce A to row-echelon form. Each time two rows 
are interchanged, multiply D by —1; each time a row is multiplied by k, multiply D by 
1/k. Do not change D when an elementary row operation of the third kind is used. 

STEP 5.3: Calculate det A as the product of D and all the diagonal elements of the row-echelon 
matrix obtained in Step 5.2. 


(See Problems 5.6 and 5.7.) This algorithm is easy to program for computer implementation; it 
becomes increasingly more efficient than expansion by cofactors as the order of A becomes larger. If 
rounding is to be used, then the pivoting strategies given in Chapter 2 are recommended. 
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INVERSION BY DETERMINANTS 


The cofactor matrix А associated with a square matrix A is obtained by replacing each element of 
A with its cofactor. If det A#0, then 
1 
lal 
If det A is zero, then A does not have an inverse. (See Problems 5.9 through 5.11 and Problems 5.18 
through 5.20.) The method given in Chapter 4 for inversion is almost always quicker than using 
(5.3), with 2 x 2 and 3 x 3 matrices being exceptions. 


at ay (5.3) 


Solved Problems 


5.1 Calculate the determinants of 
^-[} 2] e в [2 =] 


det A = 1(4) - 2(3) = -2 
det B = 2(5) - (-3)(4) = 22 


5.2 Calculate the determinant of 


expanding along (a) the first row, (b) the first column, and (c) the second column. 
(a) Expanding along the first row, we have 
det A = 4,,A,, + aA, + anA 


-$ 5 


"e «ерер 3 


КЕЛН 
72-1) (5(9) — 6(8)} + 3(—1)°{(—5)(9) — 6(7)} + 4(—1)°{(—5)(8) – 5(7)} 
= (1)(—3) + 3(—1)(—87) + 4(1)(—75) = —45 
(b) Along the first column, 
det A = a, A, +а А + а„А,, 


СРЕ з 4 
8 9.6960 56 


= X(- 1 (5(9) - 6(8)} + (—5)(—1)°{3(9) - 4(8)} + 7(—1)°{3(6) - 4(5)} 
= 2(1)(-3) + (-5)(7 (75) + 7(1)(—2) = –45 


22-1)" 


cee ee) 
з 9| *7CD 
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5.3 


5.4 


(c) Expanding along the second column gives us 


det A = @,,A,, + а„А„„+а„Ау, 


-5 6 3 4. venis] 2 4 
79 7 9/*8CD7].s g 


=3(—1){(—5)(9) - 6(7)} + 5C 1*(2(9) – 4(7)} + 8-1) ((6) - 475) 
= 3(—1)(~87) + 5(1)(710) + 8(-1)(32) = —45 


2304)? 


+5(-1)°* 


Calculate the determinant of 


by expanding along (a) the second row and (5) the third column. 
(a) Expanding along the second row gives 


scan Ж 0 -3 0 
det B= C20) $ 6l 2o 


= -2(- 1) (4(0) - 0(—8)) + 7(—1)°{(—3)(0) - (5)} + 6(—1)°{(—3)у(—8) - 4(5)) 
= -X-1)(0) + 7(1)(0) + 6(- 1)(4) = -24 


TE seca sl 


(b) Along the third column, 
det B = 0B,, + 6B,, + 0B,, = 6B,, 


-éciyn|^3 | = 6(-1)°((-3)(-8) - 4(5)) = -24 


Part b involves less computation because we expanded along a column that has mostly zeros. 


Calculate the determinant of 


1; m 25 
[а 7-855 
^73 о 8 0 
-5-1 6 9 


We expand along the third row, because it is the row or column containing the most zeros: 
det A=3A,, +0A,, +84, + 04,, = 3(7 1) М, + 8(- 1? M, 
Now we may write 
-4 2-2 
Л: x 5 
«p Ж 39 
= —4(1)(—57) + 2(-1)(68) + (-2)(1)(39) = 14 


13 


= а |+] ее у 7% 


My = 


L4 92 
md Mach d 7 |] НЕСЕТЕ Ц 


= 1(1)(68) + (—4)(—1)(61) + (-2)(1)G1) = 250 


Thus, det A = 3(1)(14) + 8(1)(250) = 2042 


45 
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5.5 Verify that det AB = det A det B (Property 5.1) for the matrices given in Problems 5.2 and 5.3. 
From the results of those problems, we know that det A det B = (—45)(-24) = 1080. Now 


23 4[-3 40 8 -3 18 
AB-|-5 5 61-2 7 6|=|35 -33 30 
789i 5 -8 0 8 12 48 


To calculate det AB, we expand along the first row, finding that 


-33 30 
12 48 


= 8(1)(—1944) + (-3)(-1)(1440) + 18(1)(684) = 1080 


det AB = &(—1)'°'| *c»c»"ps 20 жав) | 73 


12 


5.6 Use pivotal condensation to evaluate the determinant of 


022 
A=}1 0 3 
211 


We initialize D = 1 and use elementary row operations to reduce A to row-echelon form: 


әп 0з Interchanging the first and second 
ү 22 rows: № < 0(-1) = 1(-1) = -1 
211 
По 3 Adding —2 times the first row to 
02 | the third row: D remains —1 
-101 -5 


1 0 з] Multiplying the second row by 1/2: 
1 D«D(2)- -1(2)=-2 
1 


[1 3 Adding —1 times the second row 
0 1 to the third row: D remains —2 
10 -6 


1 D < р(-6) = (~2)(-6) = 12 


0 
1 
0 
[ 0 | Multiplying the third row by —1/6: 
1 
> 01 


The diagonal elements of this last matrix are all ones, so det A = D(1)(1)(1) = 12. 


5.7 Use pivotal condensation to evaluate the determinant of 


L 233 4 
42-2 5 -6 
NS Zi ож нф 
6 5 -3 6 


We initialize D = 1 and reduce A to row-echelon form: 


1 2-3 4 Adding —2 times the first row to 
шы 0 -6 Jf -M the second row: D remains 1 
=й | 6 
6. Sas 6 
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5.8 


Y A 3 4 Adding 1 times the first row to the 
0 -6 п -M third row: D remains 1 
—»|0 5-7 10 
6. $ 3 6 
Tt @ 4 Adding —6 times the first row to 
0 -6 п -14 the fourth row: D remains 1 
0 S cp 10 
SLO 7 AS = 
1 2 rl 4 Multiplying the second row by 
—|0 1 -11/6 0/6 -1/6: D+ D(-6) = 1(-6) 
0 5 z^ 10 =-6 
0: c 1$ -R 
12 -3 4 Adding —5 times the second row 
0 1 -11/6 14/6 to the third row: D remains —6 
—-|0 0 13/6 -5/3 
0 -7 15 -18 
1.3 4 Adding 7 times the second row to 
0 1 -11/6 14/6 the fourth row: D remains —6 
00 13/6 -5/3 
0010 0 13/6 -5/3 
12 awg 4 Multiplying the third row by 6/13: 
01 -11/6 14/6 D = D(13/6) = -6(13/6) = -13 
ә|0 0 1  -10/13 
0 0 13/6 -5/3 
12 -3 4 Adding —13/6 times the third row 
0 1 -11/6 14/6 to the fourth row: D remains ~13 
00 1  -10/13 
10 0 0 0 


The matrix is now in row-echelon form with diagonal elements 1, 1, 1, and 0. Thus, detA= 


—13(1)(1)(1)(0) = 0. 


Calculate the determinant of 


530 2 55 
O21 21 +6 
A=|0 121 1 
0003 6 
0001 -1 


This matrix can be partitioned into block upper triangular form with square matrices on its main 
diagonal. We introduce the partitioning 


and it follows from Eq. (5.2) that 


EE 


det A = [5] 


5.9 


5.10 


5.11 


5.12 
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Calculate the inverse of 
_[3 "i| 
ie E 4 
We shall use Eq. (5.3). Since the determinant of a 1 x 1 matrix is the element itself, we have 


Au 7 C1)"  det[4] = (1)(4) = 4 

—1)'*? det [5] = (-1)(5) = -5 
= (-1)?*' det[-1] 

Ax = (71)? det [3] = 0)3) 23 


‘The determinant of A is 3(4) — (-1)(5) = 17, so 
e f4 -5 and[41 
xo же tss 3] 


Calculate the inverse of 


In Problem 5.2, we calculated a number of cofactors for this matrix. In particular, 
A\3=-75 


and det A= —45. In addition, 


As ciy? 3-0-9) =5 


and As - ciy?|.2 3] = aes 235 
-3 8 -75 T M e^ 
Thus А=| 5-1 5 and Aang 87 -10 -32 
-2 -32 25 -5 5 25 


Find the inverse of the matrix given in Problem 5.7. 


Since the determinant of that matrix was found to be zero, the matrix does not have an inverse. 


Verify Property 5.9 for 


3 21-4 1 
A-|[2 3.0; —4 = 
ЖБ 3.58. 73 


The rank of A was determined in Problem 1.17 to be 2, so there should be at least one 2x2 
submatrix of A having a nonzero determinant. There are many, including the one in the upper left 


corner: 
| 3% 2 
23 


All 3 x 3 submatrices, obtained by deleting any two columns of A, have zero determinants. 
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5.13 


5.14 


5.15 


5.17 


Prove that the determinant of an elementary matrix of the first kind is —1. 


An elementary matrix E of the first kind is an identity matrix with two rows interchanged. The proof 
is inductive on the order of E. If E is 2x 2, then 


and det E = —1. Now assume the proposition is true for all elementary matrices of the first kind with 
order (k — 1) x (К — 1), and consider an elementary matrix E of order k x К. Find the first row of E that 
was not interchanged, and denote it as row m. Expanding by cofactors along row m yields 


detE-a,, A, ta; AS; + +а,„ „Аһ tot OA mt = A mm 
because a,,, = 0 for all j + m, and а„„ = 1. Now 
Aum = (7177 M, = Mes 


But M,,, is the determinant of an elementary matrix of the first kind having order (k — 1) х (k — 1), so 
by induction it is equal to —1. Thus, det E = A „n = My, = -1. 


Prove Property 5.3. 


If B is obtained from A by interchanging two rows of A, then B= EA, where E is an elementary 
matrix of the first kind. Using Property 5.1 and the result of Problem 5.13, we obtain 


det B = det EA = det E det A= (71) det A 
from which Property 5.3 immediately follows. 


Prove Property 5.4. 


Assume that B is obtained from an n x n matrix A by multiplying the ith row of A by the scalar k. 
Evaluating the determinant of B by expansion of cofactors along the ith row, we obtain 


det B = ka,,A,, + Ка„А„ +7 + ka, Aj, 
= k(a А + 4,.A,. tta, A, ) = k det A 


from which Property 5.4 follows. 


Prove that the determinant of an elementary matrix of the third kind is 1. 


An elementary matrix E of the third kind is an identity matrix that has been altered by adding a 
constant times one row of I to another row of I. The proof is inductive on the order of E. If E is 2 x 2, 


then 
se] oe Li 


In either case, det E = 1. Now assume the proposition is true for all elementary matrices of the third kind 
with order (k — 1) x (k — 1), and consider an elementary matrix E of order k x k. Find the first row of E. 
that was not altered from the k x k identity matrix, and denote this row as row т. The proof now 
follows that in Problem 5.13 except that here M,,,, = 1 by induction. 


Prove Property 5.5. 


If B is obtained from square matrix A by adding to one row of A a constant times another row of A, 
then B — EA, where E is an elementary matrix of the third kind. Using Property 5.1 and the results of 
Problem 5.16, we obtain 


det B = det EA = det E det А = 1 det A = det A 
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5.18 


5.19 


5.20 
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Prove that if the determinant of a matrix A is zero, then the matrix does not have an inverse. 
Assume that A does have an inverse. Then 
1 = det I = det (A 'A) = det A^' det А = det A^ '(0) =0 


which is absurd. Thus, A cannot have an inverse. 


Prove that if each element of the ith row of an n X n matrix is multiplied by the cofactor of the 
corresponding element of the kth row (i,k = 1,2,..., n; із К), then the sum of these n 
products is zero. 


For any n X n matrix A, construct a new matrix B by replacing the kth row of A with its ith row 
(i, k-1,2,..., n; ix К). The ith and kth rows of B are identical, for both are the ith row of A; it 
follows from Property 5.8 that det В = 0. Thus, evaluating det B via expansion by cofactors along its ith 
row, we may write 


07 det B= È b,B,= È а,в, (1) 
z P 


where B, is the cofactor of by. 

For each element b, (j = 1, 2,... , n) in the ith row of B, compare the submatrix B, obtained from 
B by deleting its ith row and jth column to the submatrix A,, obtained from A by deleting its kth row and 
jth column. They are the same except for the ordering of their rows; each submatrix contains all the rows 
of A except for the kth and all the columns of A except for the jth. Exactly |i — | — 1 row reorderings 
are required to make B, equal to A,,, so it follows from Property 5.3 that 


det B, = (—1)"~*'"' det A,, (2) 


These determinants are minors of B and of A, respectively, so (2) may be written in cofactor notation as 
CyB, = CIA 


В,= -Ay (3) 
Combining (1) and (3), we have 
0-9 a,B,= È a (- Ay) =- > а„А„, 
i ist i. 


which, when multiplied by —1, gives the desired result. 


Prove that A(A‘)’ = [А|І. 
Consider the (i, k) element of the product A(A*)"; it is 


Xo. k) element of (A) = 2 (a,){(k, j) element of A*) = X аА 
z ^ m 


It follows from Problem 5.19 that this sum is zero when i # k. When i = k, the sum is det A because it is 
an expansion by cofactors along the kth row of A. Therefore, we may write 


lal 


Note that if |A| #0, then A(A‘)/|A|=1, from which (5.3) follows. 
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Supplementary Problems 


In Problems 5.21 through 5.26, let 
300 
3 6 [3 | | 
A= B= С=|-1 2 0 
[3 5] 5 -6 110 


2 3 3 1-23 341 1 
D= 2. =i E=|3 51 F=| 692 5 
2 c 1 6 42 = 4 2 =$ 


5.21 Find (a) det A and (b) det B, and (c) show that det AB = det A det B. 


5.22 Find (a) det C and (b) det D, and (c) show that det CD = det C det D. 
523 Find (a) det E and (b) det F. 

5.4 Use determinants to find (а) A`’ and (b) B^". 

5.25 Use determinants to find D`', 

5.26 — Use determinants to find E^*. 


In Problems 5.27 and 5.28, find the determinant of the given matrix. 


527 201 3 sa [2 2 3 3 
10 2 2332 

112 1! 5$ 3.17 `9: 

-4 2 0 -3 3247 


5.29 Use determinants to find the inverse of the matrix given in Problem 5.28. 

5.30 Prove Property 5.8. 

5.31 Prove that if A has order n X n, then det kA = А” det A. 

5.32 Prove that if A has order n x n, then [A?| = |A[". 

5,33 Prove that if A and B are square matrices of the same order, then det AB = det BA. 
5.34 Prove that if A is invertible, then det А”! = 1/det A. 


5.35 Let A = LU be an LU decomposition of A (see Chapter 3). Show that det A is equal to the product of the 
diagonal elements of L. 


Chapter 6 


Vectors 


DIMENSION 


A vector is a matrix having either one row or one column. The number of elements in a row 
vector or a column vector is its dimension, and the elements are called components. The transpose of 
a row vector is a column vector, and vice versa. 


LINEAR DEPENDENCE AND INDEPENDENCE 


A set of m-dimensional vectors {V,,V,,...,V,,} of the same type (row or column) is linearly 
dependent if there exist constants с, €z, . . - ‚с, not all zero such that 
cV, + сї, ++ с,У, =0 (6.1) 


Example 6.1 The set of five-dimensional vectors 
{[1, 0, —2, 0, 0]", (2. 0, 3, 0, 0], [0, 2, 0, 0, 1]", and [5, 0, 4,0, 0]") 


is linearly dependent because 


1 2 0 5 0 
0 0 2 0 0 
1) -2] +2) 3] +0) 0)+(-1)} 4|] =] 0 
0 0 0 0 0 
0. 0. 1 0 0 
A set of m-dimensional vectors (V,, V,,...,V,} of the same ype is linearly independent if the 


only constants for which Eq. (6.1) holds are c, == =c 

The following algorithm may be used to determine eles a set of row vectors is linearly 
independent or dependent. The algorithm is applicable to column vectors too, if their transposes are 
considered instead. (See Problems 6.2 and 6.3.) 


STEP 6.1: Construct a matrix V whose rows are the row vectors under consideration. That is, the 
first row of V is V,, the second row of V is V,, and so on. 

STEP 6.2: Determine the rank of V. 

STEP 6.3: |f the rank of V is smaller than the number of vectors in the set under consideration 


(i.e., the number of rows of V), then the vectors are linearly dependent; otherwise, 
they are linearly independent. 


LINEAR COMBINATIONS 


A vector B is a linear combination of vectors V,, V,,..., V, if there exist constants 
d, ds... d, such that 


В= ау, +4У, dy, (6.2) 


For the matrix addition and equality of (6.2) to be defined, the vectors must all be of the same type 
(row or column) and have the same dimension. 
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Example 6.2 The vector [-3, 4, —1, 0, 2]" is a linear combination of the vectors of Example 6.1 because 


=3 1 2 0 5 
4 0 0 2 0 
-1|[20| 22|«1]3|42]0| & (-1)| 4 
0 0 0 0 0 
2 0. 0. 1 0 


Equation (6.2) represents a set of simultaneous linear equations in the unknowns 
d,,d,,...,d,. The algorithms given in Chapter 2 may be used to determine whether or not the d, 
(i9 1,2,... , n) exist and what they are. (See Problems 6.4 and 6.5.) 


PROPERTIES OF LINEARLY DEPENDENT VECTORS 


Property 6.1: Every set of т + 1 or more m-dimensional vectors of the same type (either row or 
column) is linearly dependent. 


Property 6.2: Ап ordered set of nonzero vectors is linearly dependent if and only if one vector can 
be written as a linear combination of the vectors that precede it. 


Property 6.3: If a set of vectors is linearly independent, then any subset of those vectors is also 
linearly independent. 


Property 6.4: If a set of vectors is linearly dependent, then any larger set containing this set is also 
linearly dependent. 


Property 6.5: Апу set of vectors of the same dimension that contains the zero vector is linearly 
dependent. 


Property 6.6: The set consisting of a single vector is linearly dependent if and only if that vector is 
the zero vector. 


ROW RANK AND COLUMN RANK 


Consider each row of a matrix A to be a row vector. The row rank of A is the maximum number 
of linearly independent vectors that can be formed from these row vectors; it is the rank of A (see 
Problem 6.11). Similarly, the column rank of A is the maximum number of linearly independent 
vectors that can be formed from the columns of A. It may be obtained by calculating the rank of A’, 
because the rows of A" are the columns of A. The row rank of a matrix equals its column rank (see 
Problem 6.10); so the column rank of A is also the rank of A. 
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Solved Problems 


Determine whether the set ([1, 1,3], [2, —1,3], (0, 1, 1], [4,4,3]} is linearly independent. 


Since the set contains more vectors (four) than the dimension of its member vectors (three), the 
vectors are linearly dependent by Property 6.1. They are thus not linearly independent. 


Determine whether the set ([1, 2, —1, 6], [3, 8, 9, 10], (2, — 1, 2, —2]) is linearly independent. 
Using Steps 6.1 through 6.3, we first construct 
1 k =I 6 
У= |3 8 9 10 
2-1 2 -2 


Matrix V was transformed in Problem 1.13 into the row-echelon form: 


By inspection, the rank of V is 3, which equals the number of vectors in the given set; hence, the given 
set of vectors is linearly independent. 


Determine whether the set ([3,2, 1, 4, 1]”, (2, 3,0, –1, —1]", (1, —6,3, -8, 7]") is linearly 
independent. 


Using the algorithm of this chapter, we construct 

> 21-4 1 

v=|2 30 -1 -1 

% єлї HBr 0 

which we transformed into row-echelon form in Problem 1.15: 
1 2/3 1/3 -4/3 1/3 
0 1 -2/5 1 i! 

о 0 0 0 0 


Since the rank of V is 2, which is less than the number of vectors in the given set, that set is linearly 
dependent. 


Determine whether (6, 10, 2]" is a linear combination of [1, 3, 2]", [2, 8, —1]", and [—1, 9, 2]". 


It is a linear combination if and only if there exist constants d,, d;, and d, such that 


[allel е] 


Solving this system is equivalent to solving the systems of Problem 2.8 with each x replaced by a d. In 
that problem we found that this system is consistent; hence (6, 10, —2]" is a linear combination of the 
other three vectors—in particular, for d, = 1, d, —2, and d, = - 1. 


Determine whether [5, 1, 8] is a linear combination of (2,3, 5], [1,6,7], and [0, 1, 1]. 


It is a linear combination if and only if there exist constants d,, d,, and d, such that 
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6.6 


6.7 


[5. 1,8] = d,[2, 3, 5] + d,[1,6, 7] + 4,0, 1, 1] 
= [24, + d,, 3d, + 6d, + d,, 5d, + 7d, + dj] 

which is equivalent to the system 

2d * d, =5 

3d, * 6d, * d, =1 

5d, * 7d, * d,-8 
This system was shown in Problem 2.5 to be inconsistent, so [5, 1, 8] is not a linear combination of the 
other three vectors. 


Prove that every set of m + 1 or more m-dimensional vectors of the same type (either row or 
column) is linearly dependent. 

Consider a set of n-such vectors, with n > m. Equation (6.1) generates m-homogeneous equations 
(one for each component of the vectors under consideration) іп the n-unknowns c,, c;,. .., c,. If we 
were to solve those equations by Gaussian elimination (see Chapter 2), we would find that the solution 
set has at least n — m arbitrary unknowns. Since these arbitrary unknowns may be chosen to be nonzero, 
there exists a solution set for (6.1) which is not all zero; thus the n vectors are linearly dependent. 


Prove that an elementary row operation of the first kind does not alter the row rank of a 
matrix. 


Let B be obtained from matrix A by interchanging two rows. Clearly the rows of A form the same 
set of row vectors as the rows of B; so A and B must have the same row rank. 


Prove that if AX = 0 and BX = 0 have the same solution set, then the n X n matrices A and B 
have the same column rank. 


The system AX — 0 can be written as 


а) 


ХА ХА +++ X, 
where A, is the first column of A, A, is the second column of A, and so on, and X = [x,,x,,. .., x,]". 
Similarly, the system BX = 0 can be written as 


xB, + x,B,+---+x,B,=0 (2) 


Denote the column rank of A as a, and the column rank of B as b. Assume that the column rank of 
A is greater than the column rank of B, so that а > b. Now there must exist a columns of A which are 
linearly independent. Without loss of generality, we can assume that these are the first a columns of A. 
(If not, rearrange A so that they are; this column rearrangement does not change the column rank of A, 
by reasoning analogous to that used in Problem 6.8.) However, the first a columns of B are linearly 
dependent, because b is assumed to be smaller than a. Thus, there exist constants d,, d,, . . . , d,, not all 
zero, such that 


d,B, + d,B, + + d,B, 


From this, it follows that 
d,B, + d,B,+-*'+ d,B, +0B,., * € 0B, =0 
and that 
xd, х=й, c6 o xd, Xatna 


is a solution of system (2). Since these same values are given to be a solution of system (1), it follows 
that 
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аА, + d,A,+---+d,A,=0 


where, as noted, the constants d,,d,,...,d, are not all zero. But this implies that A,,A,,...,A, are 
linearly dependent, which is a contraction. Thus the column rank of A cannot be greater than the column 
rank of B. 

A similar argument, with the roles of A and B reversed, shows that the column rank of B cannot be 
greater than the column rank of A, so the two column ranks must be equal. 


Prove that an elementary row operation of any kind does not alter the column rank of a 
matrix. 


Denote the original matrix as A, and the matrix obtained by applying an elementary row operation 
to A as B. The two homogeneous systems of equations AX = 0 and BX = 0 have the same set of solutions 
(see Chapter 2). Thus, as a result of Problem 6.8, A and B have the same column rank. 


Prove that the row rank and column rank of any matrix are identical. 


Assume that the row rank of an m X n matrix A is r, and its column rank is c. We wish to show that 
т = с. Rearrange the rows of A so that the first ғ rows are linearly independent and the remaining m ~ r 
rows are the linear combinations of the first r rows. It follows from Problems 6.7 and 6.9 that the column 


rank and row rank of A remain unaltered. Denote the rows of A as A,, A,,...,A,,, in order, and define 
A, Ln 
B- шы and C= А 3 
A, A, 


Then A is the partitioned matrix [ | Furthermore, since every row of C is a linear combination of rows 
of B, there exists a matrix T such that C = TB. In particular, if 


A,,ı =d,A, + djA, * +а,А, 


then [d,, dz, - . ..d,] is the first row of Т. Now for any n-dimensional vector X, 
-[sx]- [9x] 
ЕЕЕ" 


Hence, АХ = 0 if and only if ВХ = 0, and it follows from Problem 6.8 that A and В have the same 
column rank c. But the columns of B are r-dimensional vectors, so the column rank of B cannot be 
greater than r. That is, 


esr u) 


By repeating this reasoning on A’, we conclude that the column rank of А” cannot be greater than 
the row rank of A’. But since the columns of A' are the rows of A and vice versa, this means that the row 
rank of A cannot be greater than the column rank of A; that is, 


г=с (2) 
We conclude from (1) and (2) that г = c. 


Prove that both the row rank and the column rank of a matrix equals its rank. 


Let U be a matrix in row-echelon form obtained from A by elementary row operations. Then it 
follows from Problem 6.9, that A and U have the same column rank. Now denote the rank of A as r. 
From the definition of rank, r is the number of nonzero rows in U. Since the first nonzero element in 
each of the first r rows of U appears in a different column, it is simple to show that the first r rows of U 
are linearly independent and, therefore, that the row rank of U is r. The result of Problem 6.10 tells us 
that the column rank of U is also r. And, since U and A have the same column rank, the column and 
row-ranks of A are equal to its rank r. 
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6.13 


6.14 


Problem 6.8 suggests the following algorithm for choosing a maximal subset of linearly 
independent vectors from any given set: Construct a matrix A whose columns are the given set 
of vectors, and transform the matrix into row-echelon form U using elementary row 
operations. Then AX = 0 has the same solution set as UX = 0, which implies that any subset of 
the columns of A are linearly independent vectors if and only if the same subset of columns of 
U are linearly independent. Now the columns of U containing the first nonzero element in 
each of the nonzero rows of U are a maximal set of linearly independent column vectors for U, 
so those same columns in A are a maximal set of linearly independent column vectors for A. 

Use this algorithm to choose a maximal set of linearly independent vectors from [3, 2, 1], 
(2, 3, -6], [1,0,3], [-4, –1, -8], and [1, —1, 7]. 


We form the matrix 

3 hi He 1 

A=|2 E Жы ы: 

1, 96 $3 -8 7 

which, as shown in Problem 1.15, has the row-echelon form 
1 2/3 M3 -4/3 1/3 

U=|0 1 -2/5 1 =$ 
о 0 0 0 0 


The first and second columns of U contain the first nonzero element in each nonzero row of U. 
Therefore, the first and second columns of A constitute a maximal set of linearly independent vectors for 
the columns of A. That is, [3, 2, 1] and [2, 3, —6) are linearly independent, and all the other vectors in 
the original set are linear combinations of those two. In particular, 


[1,0,3] = 3[3,2, 1] - 3[2,3, -6] 
[—4, -1, -8] = (-2)[3, 2, 1] + (1)[2, 3, -6] 
[1,-1.7] = ([3.2, 1] + (—1)[2,3, —6] 


Choose a maximal subset of linearly independent vectors from the following set: 


EM LI BU B 


We form the matrix 


which has the row-echelon form 
1H & 1 1 
U=|0 0 1 1 12 12 
0000 0 1 


The first, third, and sixth columns of U contain the first nonzero element in each of its nonzero rows. 
Therefore, the first, third, and sixth columns of A constitute a maximal set of linearly independent 
vectors for the original set of vectors. 


Prove that an ordered set of nonzero vectors (V,, V,,..., V,) is linearly dependent if and 
only if one of its vectors can be written as a linear combination of the vectors that precede it. 
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Suppose the set is linearly dependent, and let i be the first integer between 2 and n for which 
LAE, PENAS. V,) forms a linearly dependent set. Such an integer must exist, and at the very worst i = n. 
Then there exists a set of constants d,, d,, . . . , d,, not all zero, such that 


dV,*d,V,*--*d, V ,* dV,-0 


Furthermore, d, # 0, for otherwise the set (V,, V. . 
the defining property of i. Hence, 


> V,.,) would be linearly dependent, contradicting 


d, d. d, 

Wenig quon 
That is, V, can be written as a linear combination of V, 
On the other hand, suppose that for some i (i = 2, 


V = 4,У, *dV di N 
Then dV, + dN, o di V + (DV, +0V,,, o 0v, =0 


m) 


This is (6.1) with c, = -1 #0, с, = d, (к= 1,...,i- D), апас, -0(k 2i li 2,..., n). So the set 
of vectors is linearly dependent. 


Supplementary Problems 


In Problems 6.15 through 6.20, determine whether the given set of vectors is linearly independent. 


(1, 2”, (2. 4]*) 

(ft. 1.2]. [2.2.2]. (2,2, 1} 

([1.0. 1]. [1.2.0]. (0. 1.2]) 

([1.0,2. 0]", 2, 2.0, 1]%, (1, 72.6. -1]7) 

([2.0, 1, 1], (0,1, 2, –1), [1, —1, —1, 1], (0.0, 1,2]) 


([1,2. 1]. [1,1,0], (0, 1, —1]". (2.1, 3]7} 


Is [1,3]" a linear combination of the vectors given in Problem 6.15? 


(a) Determine whether (0, 0, 1] can be written as a linear combination of the vectors given in Problem 
6.16. (b) Repeat part a for the vector [1,2,0]. 


(a) Determine whether [2, 1, 2, 1] can be written as a linear combination of the vectors given in Problem 
6.19. (b) Repeat part a for the vector [0, 0, 0, 1]. 


Show that any 3-dimensional row vector can be expressed as a linear combination of the vectors given in 
Problem 6.17. 


Choose a maximal subset of linearly independent vectors from those given in Problem 6.15. 
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6.30 


6.31 


6.32 


6.33 


6.34 


Choose a maximal subset of linearly independent vectors from those given in Problem 6.16. 


Choose a maximal set of linearly independent vectors from the following: [1,2, 1, —1], {1,0,-1,2], 
(2,2,0, 1], (3.2. -1,3], (0, 1, 1, 0], [3, 3, 0, 3}. 


An m-dimensional vector V is a convex combination of the m-dimensional vectors V,, V,,..., V, of the 
same type (row or column) if there exist nonnegative constants d,, d,, . . . , d, whose sum is 1, such that 
v=d,V, + d,V, + +++ d,V,. Show that [5/3, 5/6] is a convex combination of the vectors [1, 1], [3,0], 
and [1,2]. 


Determine whether [0, 7]" can be written as a convex combination of the vectors 
[1] Г] Gb pu 
6. 9 H 1 


Prove that if (V,, V}, . . . , V,} is linearly independent and V cannot be written as a linear combination of 
this set, then (V,, V,, ... , V,, V) is also linearly independent. 


Prove Property 6.5. 


The null space of a matrix A is the set of all vectors which are solutions of AX = 0. Determine the null 
space of 


Determine the null space of the matrix 
EE y 3] 
amfi i 


Determine the null space of the matrix 


> 
" 
Rone 
vene 
--ooc 
aoan 
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Eigenvalues and Eigenvectors 


CHARACTERISTIC EQUATION 


A nonzero column vector X is an eigenvector (or right eigenvector or right characteristic vector) of 
a square matrix A if there exists a scalar A such that 


АХ =АХ (7.1) 


Then A is an eigenvalue (or characteristic value) of A. Eigenvalues may be zero; an eigenvector may 
not be the zero vector. 


Example 7.1 [1, —1]' is an eigenvector corresponding to the eigenvalue A = —2 for the matrix 
ре 
А-2 i] 
; з 5 1]_[-2]__„/[-1 
каше [2 -Ї-]=[7=-[7] 
The characteristic equation of an n x n matrix A is the nth-degree polynomial equation 


det (A - AI) =0 (72) 


Solving the characteristic equation for A gives the eigenvalues of A, which may be real, complex, or 
multiples of each other. Once an eigenvalue is determined, it may be substituted into (7.7), and then 
that equation may be solved for the corresponding eigenvectors. (See Problems 7.1 through 7.3.) The 
polynomial det (A — AI) is called the characteristic polynomial of A. 


PROPERTIES OF EIGENVALUES AND EIGENVECTORS 


Property7.1: The sum of the eigenvalues of a matrix is equal to its trace, which is the sum of the 
elements on its main diagonal. 


Property 7.2: Eigenvectors corresponding to different eigenvalues are linearly independent. 
Property7.3: А matrix is singular if and only if it has a zero eigenvalue. 


Property 7.4: If X is an eigenvector of A corresponding to the eigenvalue A and A is invertible, then 
X is an eigenvector of A ' corresponding to its eigenvalue 1/A. 


Property 7.5: If X is an eigenvector of a matrix, then so too is kX for any nonzero constant k, and 
both X and kX correspond to the same eigenvalue. 


Property 7.6: A matrix and its transpose have the same eigenvalues. 


Property 7.7: The eigenvalues of an upper or lower triangular matrix are the elements on its main 
diagonal. 


Property 7.8: The product of the eigenvalues (counting multiplicities) of a matrix equals the 
determinant of the matrix. 


Property 7.9: If X is an eigenvector of A corresponding to eigenvalue A, then X Is an an eigenvector 
of A — cl corresponding to the eigenvalue A — c for any scalar c. 
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LINEARLY INDEPENDENT EIGENVECTORS 


The eigenvectors corresponding to a particular eigenvalue contain one or more arbitrary scalars. 
(See Problems 7.1 through 7.3.) The number of arbitrary scalars is the number of linearly 
independent eigenvectors associated with that eigenvalue. To obtain a maximal set of linearly 
independent eigenvectors corresponding to an eigenvalue, sequentially set each of these arbitrary 
scalars equal to a convenient nonzero number (usually chosen to avoid fractions) with all other 
arbitrary scalars set equal to zero. It follows from Property 7.2 that when the sets corresponding to 
all the eigenvalues are combined, the result is a maximal set of linearly independent eigenvectors for 
the matrix. (See Problems 7.4 through 7.6.) 


COMPUTATIONAL CONSIDERATIONS 


There are no theoretical difficulties in determining eigenvalues, but there are practical ones. 
First, evaluating the determinant in (7.2) for an n X n matrix requires approximately п! multiplica- 
tions, which for large n is a prohibitive number. Second, obtaining the roots of a general 
Characteristic polynomial poses an intractable algebraic problem. Consequently, numerical algor- 
ithms are employed for determining the eigenvalues of large matrices (see Chapters 19 and 20). 


THE CAYLEY-HAMILTON THEOREM 
Theorem 7.1: Every square matrix satisfies its own characteristic equation. That is, if 
det (A — А) = БА” + БА"! +--+ b! + BAF by 
then b,A" +b, AT +++ b + BA bul =0 
(See Problems 7.15 through 7.17.) 


Solved Problems 


7. Determine the eigenvalues and eigenvectors of 
o 23: - 3] 
^=[; 2] 


^-и=[_; Seo Р Lu] 
hence det (A - А) =(3 — A)(—4 — А) - 5(-2) =a? +à -2 


The characteristic equation of A is A? + A— ; when solved for A, it gives the two eigenvalues A = 1 
and А = —2. As a check, we utilize Property 7.1: The trace of A is 3 + (—4) = —1, which is also the sum 
of the eigenvalues. 

The eigenvectors corresponding to A = 1 are obtained by solving Eq. (7.1) for X = [x,. x;]" with this 
value of A. After substituting and rearranging, we have 


(2 - EI 
i [2 ЙЕ 


which is equivalent to the set of linear equations 


For this matrix, 


о] 
o] 
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2x, +5x,=0 
-2x, - 5x, 20 
The solution to this system is x, = — 3x, with x, arbitrary, so the eigenvectors corresponding to А = 1 аге 


IH Ter 
(L1 seo E-E 
Li ШЕШ 


which is equivalent to the set of linear equations 


Sx, + 5х, =0 
-2x,-2x,=0 
The solution to this system is x, = —x, with x, arbitrary, so the eigenvectors corresponding to A = —2 are 


with x, arbitrary. 


Determine the eigenvalues and eigenvectors of 


For this matrix, 
522 100] [5-4 2 2 
A-a-|3 6 3|-40 1 0]-| 3 6-a 3 
669 00 1 6 6 9-A 
The determinant of this last matrix may be obtained by expansion by cofactors (see Chapter 5); it is 
А? + 20А? - 93A + 1267 -( A – 3 (A ~ 14) 


The characteristic equation of A is —(A —3)'(A — 14) = 0, which has as its solution the eigenvalue A = 3 
of multiplicity two and the eigenvalue A — 14 of multiplicity one. As a check, we utilize Property 7.1: The 
trace of A is 5 + 6 + 9 = 20, which equals the sum of the three eigenvalues. 

The eigenvectors corresponding to А = 3 are obtained by solving (7.1) for X =[x,, х,, x,]" with this 
value of A. Thus, we may write 


(i PETS 
d REED 


which is equivalent to the set of linear equations 
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2x, + 2x, +2x,=0 
3x, +3x, +3x,=0 
6x, + 6x, + 6x, 70 


The solution to this system is x, = —x, — x, with x, and x, arbitrary; the eigenvectors corresponding to 


A=3 are thus 
Ж, 5—25 =f -f 
х=|х.[=| ох, ех [+ 0 
ху ху 0 1 
with x, and x, arbitrary. 
When A = 14, (7.1) becomes 


Sra 1 0 о] 
LEN 
669 001 Ху, 
-9 2 215 
; Reece 
6 6 —SJL*s 


which is equivalent to the set of linear equations 
-9x, + 2x, +2x,=0 
Зх, – 8x, + 3x, 70 
бх, + 6х, —Sx,=0 


ела Wo 


0 
0 
0 


The solution to this system is x, = jx, and x, = 1x, with x, arbitrary; the eigenvectors corresponding to 
А = M are thus 
ч ix, 1/3 
Х= |х. |=| }x, |= x| 1/2 
x, EX 1 


Determine the eigenvalues and eigenvectors of 
8 4] 
a=[3 - 


^-a-[ 5 s] do |Р assu] 
hence det (A - M) = (3- A(-5- А) -4(-5) = A? «244 5 


The characteristic equation of A is A7+2A+5=0; when solved for A, it gives the two complex 
eigenvalues A = —1 + i2 and A= —1 — i2. As a check, we note that the trace of A is —2, which is the sum 
of these eigenvalues. 

The eigenvectors corresponding to A — 1 are obtained by solving Eq. (7.1) for X 
value of A. After substituting and rearranging, we have 


(L5 -J-e THLE) -[0) 
ed x а] 


which is equivalent to the set of linear equations 


with x, arbitrary. 


For this matrix, 


x, x,]" with this 


7.4 


T5 
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(4- 22)х, + 4х,=0 
-5x,*(-4- 2) х,=0 
The solution to this system is x, = (—4/5 — i2/5)x, with x, arbitrary; the eigenvectors corresponding to 
= -1 + i2 are thus 


RE [5]- [Fuss pie -x[ 15: ai] 


х x; 


with x, arbitrary. 
With А = —1— i2, the corresponding eigenvectors are found in a similar manner to be 


x- z]- [С Map =x -415 i 25] 


with x, arbitrary. 


Choose a maximal set of linearly independent eigenvectors for the matrix given in Problem 
7.2. 


The eigenvectors associated with А = 3 were found in Problem 7.2 to be 


-1 эй, 
-| | + -f ] X4, X, arbitrary 
0. 1 


There are two linearly independent eigenvectors associated with A = 3, one for each arbitrary scalar. One 
of them may be obtained by setting x, = 1, x, = 0; the other, by setting x, = 0, x, =1. 
The eigenvectors associated with A = 14 are 


1/3 
x) 1/2| х, arbitrary 
1 


Since there is only one arbitrary constant here, there is only one linearly independent eigenvector 
associated with A = 14. It may be obtained by choosing x, to be any nonzero scalar. A convenient choice, 
to avoid fractions, is x, = 6. Combining the linearly independent eigenvectors corresponding to the two 


eigenvalues, we obtain 
zz - 2 
1 0 3 
0 1 6 


as a maximal set of linearly independent eigenvectors for the matrix. 


Choose a maximal set of linearly independent eigenvectors for the matrix given in Problem 
7.1. 


The eigenvectors corresponding to A = 1 were found in Problem 7.1 to be 
x xe] x, arbitrary 
Since there is only one arbitrary scalar, there is only one linearly independent eigenvector associated 
with this eigenvalue. It may be obtained by choosing x, to be any nonzero scalar. A convenient choice, 


to avoid fractions, is x, = 2. 
The eigenvectors corresponding to A = —2 are 


ч] Al x, arbitrary 
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7.6 


77 


7.8 


There is one linearly independent eigenvector associated with this eigenvalue, and it may be obtained by 
choosing x, to be any nonzero scalar. A convenient choice here is х, = 1. Collecting the linearly 
independent eigenvectors for the two eigenvalues, we have 


(3 Ul 


as a maximal set of linearly independent eigenvectors for the matrix. 


Choose a maximal set of linearly independent eigenvectors for the matrix 


oowoco 
© 
© 


осооюе 


Since this matrix is upper triangular its eigenvalues аге the elements on its main diagonal. Thus, 
А = 2 is an eigenvalue of multiplicity five. The eigenvectors associated with this eigenvalue are 


xy 1 0 0 
0 0 0 0 
ху 0|*x, 1|*x[ 0 
х, 0 0 1 
0 0. 0. 0 


with ху, ху, and x, arbitrary. Because there are three arbitrary scalars, there are three linearly 
independent eigenvectors associated with А. One may be obtained by setting x, = 1, x, = x, = 0; another 
by setting x, = 1, x, = x, = 0; and the third by setting x, = 1 and x, = x, = 0. Note that this matrix has 
only three linearly independent eigenvectors, even though it has order 5 x 5. 


Show if A is an eigenvalue of a matrix A, then it is a solution to (7.2). 

If A is an eigenvalue of A, there must exist a nonzero vector X such that AX = AX. Thus, 
AX — AX = 0, and (A — AI)X = 0. This implies that A — АГ is singular, for otherwise X = (A — AI) '0 = 0, 
which is not the case. But if A — AI is singular, then det (A — AI) = 0 (see Chapter 5). 


Show that eigenvectors corresponding to different eigenvalues are linearly independent. 


Let À,, Az,..., À, be different eigenvalues of a matrix A, and let X,,X;,...,X,, be associated 
eigenvectors. We must show that the only solution to 
CX, + 6X Heet Cn Xn = 0 (1) 
is су = c, 7-7: = с„ = 0. Multiplying (1) on the left by А, we obtain 


C,AX, + c,AX, +  * c, AX, = A0=0 
Since each vector here is an eigenvector, we use (7.1) to write 
с,А,Х, + с,А,Х, +--+ c,A,X, 70 (2) 
Multiplying (2) on the left by A and again using (7.1), we obtain 
с,АЗХ, AX, 8 8 6,AZX, =0 (3) 


Equations (1) through (3) are the first three equations of the set 


7.9 


7.10 


7.M 


7.12 
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cX, + с,Х, ++. 0 
C€AX, ECA X, 6o t6AQX, =O 
CAK, +с,АХ,‚, eX, =0 
CAK +¢,A3K, ++-++¢,,A2X,, 0 


«А "Ж + с, А ХС АТХ, 


generated by sequentially multiplying each equation оп the left by A. This system сап be written іп the 
matrix form 


(4) 


еее 


0 
The first matrix on the left is ап m х m matrix which we shall denote as Q. Its determinant is called the 
Vandermonde determinant and is 

(Ag АСА А СА 7 AGO, 7 А,)(А, 7 A047 А,) Am 7A) 


which is not zero in this situation because all the eigenvalues are different. As a result Q is nonsingular, 
and the system (4) can be written as 


eX, 0 о 
cX, 0 0 
6X \=О 0 |=| 0 
c X, 0 ° 
It follows that c,X, = 0 (i= 1,2, . . . , m). But since each X, is an eigenvector, it is not zero; so c, = 0 for 


each i. 


Prove that a matrix is singular if and only if it has a zero eigenvalue. 


A matrix A has a zero eigenvalue if and only if det (А — 01) 2 0, which is true if and only if 
det A = 0, which in turn is true if and only if A is singular (see Chapter 5). 


Prove that if X is an eigenvector corresponding to the eigenvalue A of an invertible matrix A, 
then X is an eigenvector of A`" corresponding to its eigenvalue 1/A. 


It follows from Problem 7.9 that A#0. We are given AX = AX, so A^ (AX) - A^ (AX) and 
X = АА 'X). Dividing by A, we obtain A^'X = (1/A)X, which implies the desired result. 


Prove that a matrix and its transpose have the same eigenvalues. 
If A is an eigenvalue of A, then 
0 = det (A – AI) = det ((A")' — AI”) = det(A" — AI)? = det (A7 — AI) 
by Property 5.7. Thus, А is also an eigenvalue of A”. 


Prove that if X,,X,,...,X, are all eigenvectors of a matrix A corresponding to the same 
eigenvalue A, then any nonzero linear combination of these vectors is also an eigenvector of A 
corresponding to A. 


Set X= d,X, + d,X, +-+-  d,X,. Then 
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7.13 


7.14 


7.15 


AX = A(d,X, + dX, + +d,X,„) 
= d, AX, + djAX, +--+ +d, AX, 
= d,AX, + d,AX, t +d, AX, 
= жах, * d,X, * + d,X,) = AX 
Thus, X is an eigenvector of A. Note that a nonzero constant times an eigenvector is also an eigenvector 
corresponding to the same eigenvalue. 


A left eigenvector of a matrix A is a nonzero row vector X having the property that XA = AX 
or, equivalently, that 


X(A- А) =0 а) 


for some scalar A. Again А is an eigenvalue for A, and it is found as before. Once А is determined, it is 
substituted into (1) and then that equation is solved for X. Find the eigenvalues and left eigenvectors for 


^L: 4] 


The eigenvalues were found in Problem 7.1 to be A= 1 and A = -2. Set X (X). With A= 1, (1) 


becomes 
з 5 10). 
wolls -illo taoa 
2 5 
or кы] E 2] = tor 
or [2x, - 2x,, 5x, ~ 5х,] = [0,0] 
which is equivalent to the set of equations 
2x, -2x,70 
Sx, – 5x, =0 


The solution to this system is x, = x,, with x, arbitrary. The left eigenvectors corresponding to A = 1 are 
thus [x,, x;] =[x,, x;] =x,[1, 1] with x, arbitrary. 
For A 2, (1) reduces to 


Бы 5 3]- 10.01 


or [5x, = 2x,, 5x, – 2х,] = (0. 0] 
which is equivalent to the set of equations 

5x, 721,20 

5x, 72x, 70 


The solution to this system is x, = 3x,, with x, arbitrary. The left eigenvectors corresponding to A = 
are [x,, x,] = [2x,, x,] = x,[2/5, 1] with x, arbitrary. 


Prove that the transpose of a right eigenvector of A is a left eigenvector of A’ corresponding 
to the same eigenvalue. 


If X is a right eigenvector of A corresponding to the eigenvalue A, then AX = AX. Taking the 
transpose of both sides of this equation, we obtain XA” = AX’. 


Verify the Cayley-Hamilton theorem for 


һ-[2 4] 
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The characteristic equation for A was determined in Problem 7.1 to be A? + A — 2 - 0. Substituting 


A for A, we obtain 
мА [i PIDE 4] J-B 4 


7.46 Verify the Cayley-Hamilton theorem for 


The characteristic equation for A was found in Problem 7.2 to be —A' + 20A? — 93A + 126 = 0. 
Therefore, we evaluate 


521 494 494 43 34 34 s 2. 
АЎ +2042 – 93А 4126 - -| 741 768  741|«20 51 60 Si1|-933 6 3 


1,482 1,482 1,509 102 102 111 669 


101 000 
+1260 1 O}=|0 0 0 
oot 000 


7.37 Prove the Cayley-Hamilton theorem. 


We denote the characteristic polynomial of an n X n matrix A as 


d(A) = b,A" + b, A"! e DAT + BLA + ba qd) 
and set C=A-Al (2) 
Then А) = det(A — AI) = det € (3) 


Since С is an n X n matrix having first-degree polynomials іп А for its diagonal elements and scalars 
elsewhere, it follows that the cofactor matrix C' associated with C (see Chapter 5) will have elements 
that are polynomials of degree n — 1 or n – 2 in А. Elements on the main diagonal of C' will be 
polynomials of degree n — 1; all other elements will be polynomials of degree n — 2. The same will be 
true of the transpose of this cofactor matrix; hence (C')" may be written as the sum of products of 
distinct powers of A and scalar matrices M,: 


(C)' = М, (АМ, A 7 4 & M, A * M, (4) 


where М, M,,. .., M, , are all n X n scalar matrices. 
It follows from Problem 5.20 and (3) that 


C(C')' = (det C) = d(A)I (5) 


Using (2), we obtain 
C(C^)' = (А - АКС)” = A(C')' - (C')" 
With (5), this yields 
d(A)I = A(C*)' - A(C')" (6) 
Substituting (1) and (4) into (6), we obtain 
BAL + b, [АТЖ BAD + Б] = AM, [АТ + AM, A"? ++. + AM, A+ AM, 
= М, 1А" М, ;,à ^! --- МАМА 


Both sides of this matrix equation are polynomials іп А. Since two matrix polynomials are equal if 
and only if their corresponding coefficients are equal, it follows that 
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b,1=-M,_, 
b, 1 AM, _,-M,-2 
b, 17 AM, 7M, 


b - AM, - M, 

bl = AM, 
Multiplying the first or these equations by A", the second by A'^*, the third by A", and so on (the last 
equation will be multiplied by А° = I) and then adding, we find that terms on the right side cancel, 
leaving 


b,A" +b, ,A 7 +b, ATI ee bA ТЬЕ 0 


which is the Cayley-Hamilton theorem for A with characteristic polynomial given by (1). 


Supplementary Problems 


In Problems 7.18 through 7.26, find the eigenvalues and corresponding eigenvectors for the given matrix 


7.18 isl 749 P zl im А H 


13 4 
-8 7 Eg 35 

nm [4i cmn [2 3) тә [11] 
paru 0 1-1 2 2 3 

734 |0 2 i| 725 |o -i 1| 726 | 216 
00 3 0 0 0 366 


In Problems 7.27 through 7.34, find the eigenvalues and a maximal set of linearly independent eigenvectors 
for the given matrix 


510 510 500 
ът |05 1] 7233 |05 01 72 |o 5 0 
005 005 005 
[3 100 3100 4 3 0 0 
0300 0310 -2-1 0 0 
TM: 16.0.3 af ТИ ood a) 7X [à w 4 3 
lo 0 0 3 0003 o 0 -3 -2 


In Problems 7.35 through 7.40, find the eigenvalues and a maximal set of linearly independent left 
cigenvectors for the given matrix 


7.35 The matrix in Problem 7.18. 


7.36 The matrix in Problem 7.19. "n 


7.37 The matrix in Problem 7.20. 
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3 2} E 1 1 E s-t 1 
7.38 2 3 7.39 1 ci 7.40 ae E! 
=F = 4 = Ж. 1 E -i 3 


7.41 Verify the Cayley-Hamilton theorem for the matrix in (а) Problem 7.18; (b) Problem 7.24; and (с) 
Problem 7.30. 


7.42 Show that if A is an eigenvalue of A with corresponding eigenvector X, then X is also an eigenvector of 
А? corresponding to A*. 


7.43 Show that if A is an eigenvalue of A with corresponding eigenvector X, then for any scalar c, X is an 
eigenvector of A — cI corresponding to the eigenvalue A — c. 


7.44 Prove that if A has order n x n, then 
det(A — Al) = (1) (A^ — (trace A)A”~' + O(A^7)) 


where O(A" ^?) denotes a polynomial in A of degree n — 2 or less. 
7.45 Prove that the trace of a square matrix is equal to the sum of the eigenvalues of that matrix. 
7.46 Prove that trace (A + B) = trace A + trace B if A and B are square matrices of the same order. 
7.471 Prove that trace AB = trace BA if A and B are square matrices of the same order. 
7.48 Show that if S is an invertible matrix of the same order as A then trace(S 'AS) = trace A. 
7.49 Prove that the determinant of a square matrix equals the product of all the eigenvalues of that matrix. 


7.50 Show that the n x n matrix 


has as its characteristic equation 
(-1)"(A" +a, A" a, A 7 * + a +a À +a) =0 


The matrix C is called the companion matrix for this characteristic equation. 


Chapter 8 


Functions of Matrices 


SEQUENCES AND SERIES OF MATRICES 


A sequence, (B,) of matrices B, = [Б], all of the same order, converges to a matrix В = [b,] if 
the elements be ! converge to b, for every i and j. The infinite series E, B, converges to B if the 
sequence of partial sums (S, = $t. B, ) converges to B. (See Problem 8.1.) 


WELL-DEFINED FUNCTIONS 
If a function f(z) of a complex variable z has a Maclaurin series expansion 


Ло) = È az 


n=0 


which converges for |z| < R, then the matrix series Ežo a, A" converges, provided A is square and 
each of its eigenvalues has absolute value less than R. In such a case, f(A) is defined as 


ЛА) = У a, 


and is called a well-defined function. By convention, A? = 1. (See Problems 8.2 and 8.3.) 


Example 8.1 
1 1 el 
esirgztj Laz! 


converges for all values of z (that is, А = ®). Since every eigenvalue A of any square matrix satisfies the 
condition that |A| <=, 


ote daa vcs) 


is well defined for all square matrices A. 


COMPUTING FUNCTIONS OF MATRICES 


An infinite series expansion for f(A) is not generally useful for computing the elements of f(A). It 
follows (with some effort) from the Cayley-Hamilton theorem that every well-defined function of an 
nX n matrix A can be expressed as a polynomial of degree n — 1 in A. Thus, 


ДА) = a, А" + а„_,А"? * t aA! + a, A + aol (8.1) 
where the scalars a, ,, à, 5,...,45, 4,, à, are determined as follows: 
STEP 8.1: Let 
ҚА) =а, 1А a, 3A 7 * aj + aA t ay 
which is the right side of (8.1) with A’ replaced by A’ (j=0,1,...,—1). 
STEP 8.2: For each distinct eigenvalue A, of A, formulate the equation 


FADS (A) (82) 
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STEP 8.3: |f A, is an eigenvalue of multiplicity k, for k>1, then formulate also the following 
equations, involving derivatives of f(A) and r(A) with respect to А: 
ГО = Olas, 
ГО) = "ODE, (83) 


STEP 8.4: Solve the set of all equations obtained in Steps 8.2 and 8.3 for the unknown scalars 
aos Ays.» Ay ye 
Once the scalars determined in Step 8.4 are substituted into (8.1), f(A) may be 
calculated. (See Problems 8.4 through 8.6.) 


THE FUNCTION e^ 


For any constant square matrix A and real variable t, the matrix function e^' is computed by 
setting В = Ar and then calculating e" as described іп the preceding section. (See Problems 8.7 
through 8.10.) 

The eigenvalues of В = At are the eigenvalues of A multiplied by t (see Property 7.5). Note that 
(8.3) involves derivatives with respect to A and not г; the correct sequence of steps is to first take the 
necessary derivatives of f(A) and r(A) with respect to A and then substitute А = A,. The reverse 
procedure—first substituting A = А, (a function of t) into (8.2) and then taking derivatives with 
respect to t—can give erroneous results. 


DIFFERENTIATION AND INTEGRATION OF MATRICES 


The derivative of А = (a;] is the matrix obtained by differentiating each element of A; that is, 
dA! dt = [|da dt). Similarly, the integral of A, either definite or indefinite, is obtained by integrating 
each element of A. Thus, 


Piafa] aw [хаа] 


(See Problems 8.11 and 8.12.) 


DIFFERENTIAL EQUATIONS 
The initial-value matrix differential equation 


X(t) = AX() *F() X(t) =C 
has the solution 


xq) = eMC «e f e^" F(s) ds (84) 


or, equivalently, 


X(t) = e^ c + [ e^ P F(s) ds (8.5) 


If the differential equation is homogeneous [i.e., F(t) = 0], then (8.4) and (8.5) reduce to 
X(t) = e^ "c, 
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In (8.4) and (8.5), the matrices e^ ^ е^, and e^"? are easily computed from е^ by 
replacing the variable t with г — tọ, —s, and t — s, respectively. Usually, X(t) is obtained more easily 
from (8.5) than from (8.4), because the former involves one fewer matrix multiplication. However, 
the integrals arising in (8.5) are generally more difficult to evaluate that those in (8.4). (See 
Problems 8.13 and 8.14.) 


THE MATRIX EQUATION AX + XB- C 
The equation AX + XB = C, where A, B, and C denote constant square matrices of the same 


order, has a unique solution if and only if A and B have no eigenvalues in common. This unique 
solution is given by 

X=- [ e^ Ce" dt (8.6) 
provided the integral exists (see Problem 8.15). 


Example 8.2 For A = I and B = 0 the matrix equation has the unique solution X = C, but the integral (8.6) 
diverges. 


Solved Problems 


8. Determine lim B, when 


1 2+k 

k? 3+2k 

B, aie 

5 (1+2) 

k 
: » liu Dd do uem " nu 
Since fim ga=0 Базуд-; dms-5 ad fim (1+ 7) =e 

т te 

weihave jim, = [0 12] 


8.2 For which matrices А is the function cos A well defined? 


The Maclaurin series for cos z is 


z 
cosz=1- 57 + 


which converges for all values of z (that is, А =~). Every eigenvalue of any square matrix satisfies the 
condition that [A] < 2, so 
(Ay 

(2n)! 


is well defined for every square matrix A. 
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8.3 


8.4 


8.5 
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Determine whether arctan A is well defined for 
_[2 | 
oa eas 


атеке Р Ena СОЛ 
меле Зо Т 2, antl 


The Maclaurin series for arctan z is 


which converges for all values of z having absolute value less than 1. Therefore, 


A NT ee e CciyA"" 
amanAcA- qp 6-00 У үү 


is well defined for any square matrix whose eigenvalues are all less than 1 in absolute value. The given 
matrix A has eigenvalues A, = 0 and A, = 4. Since the second of these eigenvalues has absolute value 
greater than 1, arctan A is not defined for this matrix. 


Find cos A for the matrix given in Problem 8.3. 


From Problem 8.2 we know that cos A is well defined for all matrices. For this particular 2 x 2 
matrix A, (8.1) becomes 


2a,+a, 4a, ] 


cos A = дА + a a, 2a, + a, 


а) 
Now f(A) = cos А, r(A) = a,A + a,, and the distinct eigenvalues of A are A, = 0 and A, = 4. Substituting 
these quantities into (8.2) once for each distinct eigenvalue, we formulate the two equations 

cos 0 = а,(0) + а, 

cos 4 = a,(4) + а, 


Solving these equations for a, and a,, we obtain а, = соѕ0 = 1 and a, = (cos4 – 1)/4 = —0.413411. 
Substituting these values into (1) and simplifying give us 


0.173178 – | 


сел [батын Оз 


Find е^ for the matrix given in Problem 8.3. 


It follows from Example 8.1 that e^ is defined for all matrices. For this particular 2 x 2 matrix A, 
(8.1) becomes 


2a,*a, 4a, ] a) 


Asat ad= | a, 2a, + à, 


Now f(A) = е", r(A) = аА + a, and the distinct eigenvalues of A are A, =0 and A, =4, Substituting 
these quantities into (8.2) once for each eigenvalue, we formulate the two equations 


a,(0) +a, 
e'-a(4)*a, 


Thus a, = e" = 1 and a, = (e* — 1)/4 = 13.3995. Substituting these values into (1) and simplifying give us 


27.7991 53.5982 


cmm [ 13.3995 27.7991 
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8.7 


Find sin A for 


“> 2 0 
A-| 0-2 1 
G 0: <2 


The Maclaurin series for sin z converges for all finite values of z, so sin A is well defined for all 
matrices. For the given matrix 3 x 3 (8.1) becomes 


4-8 2 -2 2 0 100 
sinA=a,A *aA*ad-a|0 4 -4|*a| 0 -2 1|+а/0 1 0 


0 0 4 0 0-2 001 w 
4a,-2a,+a, —8a,+2a, 2a, 
= 0 4a, – 2а, * ay —4a,* a, 
0 0 4а,-— 2a, + a, 
Matrix A has eigenvalue А = —2 with multiplicity three, so we will have to use Step 8.3. We determine 
ҚА) =sin à ҚА) = a + a,A+ ay 
f'(A) = cos A r(4) 224,4 +a, 


SA) = -sin à (А) =2a, 

and write (8.2) and (8.3) as, respectively, 

sin (-2) = a,(-2)? + a,(-2) + a, 

cos(-2) =2a,(—2) + a, 

—sin(-2) = 2a, 
We thus obtain a, = — } sin(-2) = 0.454649; a, = cos (-2) — 2 sin (-2) = 1.40245; and a, = 2 cos (-2) - 
sin (2) = 0.0770038. Substituting these values into (1) and simplifying give us 
—0.909297 -0.832294 0.909297 
sinA= 


0 —0.909297 —0.416147 
0 0 —0.909297 


Find e^' for 
го 1] 
^=[1 à 
We set 
[9t 
ААЙЫ d o] 


and compute e”. Since B is of order 2 x 2, (8.1) becomes 


= aB+ad=[ a Ei a) 


-a,i ay 


Here f(A) = e`, r(A) = a,A + ao, and the distinct eigenvalues of B are A, = it and A, = — it. Substituting 
these quantities into (8.2) separately for each eigenvalue, we obtain the two equations 
e" =a,it+ a, 
e“=a,(-it) +a, 


Solving these equations for a, and a,, we obtain 
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A. 
2it 


a, e-e) ER and а= ettet) = cost 


Substituting these values into (1), we determine 


e^ 


[set ar 
—sint cost 


Find e^' for 


a-[s 2] 


cer. 4$ 
вел [а Al 


and compute e". Since B is of order 2 x 2, (8.1) becomes 


в ан 
Bat —2аи+а, 


We set 


а) 


Here f(A) = e^, (А) = a,À + ao, and the distinct eigenvalues of В are A, = 21 and А, 
these quantities into (8.2) once for each eigenvalue, we obtain the two equations 


= 4. Substituting 


e” =a (2t) + a, 
e “= a,(—41) + ag 


Solving these equations for a, and ay, we obtain a, = (е — е “)/6t and a, = (2е + e^ ")/3. Substitut- 
ing these values into (1), we get 


eval [E +2e“ e-e“ | 
618e'-8e* 2e" + дег“ 


Find e^' for 


We set B = At and compute e”. Since B is a 3x 3 matrix, (8.1) becomes 


а, 0 0 
а,В? + a,B + a,l = at а, 0 a) 
a? tat 0 aftatta, 


Now f(A) = e", r(À)= a;A? + a,À + ap, and the distinct eigenvalues of B are A, = 0 with multiplicity two 
and A,=1 with multiplicity one. Substituting these quantities, along with f'(A)- e^ and r'(A)= 
2a, + a,, into (8.2) and (8.3), we formulate the three equations 


e'=a,f +a,t+a, 


Thus. a, = (e ~t- 1), a, - 1, and a, — 1. Substituting these values into (1) and simplifying, we 


obtain 
1 0 0 
t 10 
е-1 0 e 
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8.10 Establish the equations that are needed to find е^ if 


123456 
012345 
л-|002345 
000234 
000000 
000001 


We set В = At and compute e. Since B is a 6 x 6 matrix (8.1) becomes 


e” = a,B* + a,B* + а,В? + a,B? + a,B + agl (2) 


The distinct eigenvalues of В are A, = г with multiplicity three, А, = 21 with multiplicity two, and A, = 0 
with multiplicity one. We determine 
КА) = е" А) =а,А? +а,А+а,А + аА? + а,А + ay 
РОА) = e> т'(Ау=5а,А* + да + 3a,À + 2а,А + a, 
f(a = е^ (А) = 20a, A? + 12а,А? + 6a,A + 2a, 
and (8.2) and (8.3) become 
e a au! tartar + аи +а, 
e' = Sat“ + 4a,U + Зам? + 2a,t+ a, 
e' = 20a,0 + 12a, + бам + 2a, 
e" = a (20) + a (20) + at)! + а (20) + а (20) + ay 
e" = 5а,(21)* + 4a (20)! + 3a (2t) + 2a, (28) + a, 
e° = а,(0) + a,(0)* + а,(0) + а,(0) + a,(0) + ay 
which should be simplified before they are solved. 


8.14 Find dA/dt if 
NES | 
ae [ sint 45 


dia LT 
dA _ a U0*D gle Tm “| 
а |а. 4 
à шет) 5, (45) 


8.12 Find f A dt for A as given in Problem 8.11. 


(^ * 1) dt | ? dt 

faa f M FORT озаже, dette, 
m —€0st* c, 451+ c, 
sin t dt 45 dt 


8.13 Solve X(t) = AX(r) + F(r) with initial value X(0) = C when 


[t2] vede] е-[] 


The solution is given by either (8.4) or (8.5). We shall use (8.4) here, and (8.5) in Problem 8.14. 
For A as given, e™ has already been calculated in Problem 8.8. Therefore, we can compute 
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А обоо рн 
dM api es 
ee EES oe 
om 4e" +2e% ег? 
к) из» lI 
[а-а 


Í eM E(s)ds = 


30 
1 | (ае «2e *y(-5e 
180 


(Be - 8e ")(-5Se^ 


6e' + 10е — 467“ 


Thus 


TEE ie")ds 


"Aires e-e6t I -Se'- e" +6 ] 
Be” -8e" 2e" eae 


-xl 


-10e* 


*4e" +6. 


-10e + 4e% + 6. 


| 


t-e" 46) 4 (E? - e - 1067 + de + 6) 
"= е“ 46) + 2e" + 4e )(- 1007 + 4e +6) 


бе + Se" + е7“ | 


X() = е^ C+ ef e^" F(s)ds 


ZR wl seed ses 
-4e * 6e' + 10e” — 4e" 


8.14 Use (8.5) to solve Problem 8.13. 


-u 
he 


The vector e^ "'C was found in Problem 8.13. Furthermore, 


розу Lf dene? «26707 


a-n 


e e 


fe ACO OR(S) ds = 


-2e + gey 


Thus, 


12. TE ie XM 


as before. 


[еее 


ОУ 


m 


eiie 
2e" - iet 


1 (cet есе өү, La ote eta de 
6 6L -2e + 2e” - ie 


рее ден 
-We le- de 


2-4 


=ar 


E 


+e" 


[RÀ “gje de 


e 


-de 


| 


(Quod „ае 


e 
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8.15 Solve the matrix equation AX + XB=C for X when 


[$2] eB «129 


In preparation for the use of (8.6), we calculate 


о [ее o ьо [бе Деи е ет, 
е -[ ШУ and e™ = je фет de den 
Tui “сг irem бее" ] 
"Nr LN -10/7 -4/7 
ind S if Breed ree [ 7/27 34] 
8.16 Prove that e“‘e™ = e'^**" if and only if the matrices A and B commute (that is, if and only if 


the commutative property for multiplication holds for A and B). 


If AB = BA, and only then, we have 


(А + В)? = (А + В)(А + B) = А? + АВ + ВА + B? = A! + 2AB+B’ 


2 
2$ оу. 
-E(D« LÀ 
and, in general, (A+B)"= X (D EE ж (1) 
e 
where (Q- TE D 


is the binomial coefficient (“л things taken k at a time"). 
Now according to the defining equation, we have for any A and B: 


(5, 1 ке)(5, i BY) = > Ў n Lia 


n! 130070 (n - k)! 
ara AB а ^ 
-E[£ o [Зх] о 
and gs E (+BY У («By г (3) 


The last series in (3) is equal to the last series in (2) if and only if (1) holds; that is, if and only if A and 
B commute. 


8.17 Prove that e*"e^ ^ = eA, 


Setting г = 1 in Problem 8.16, we conclude that е^е* = е(^* 9) if A and B commute. But the matrices 
At and — As commute, since 
(At)(—As) = (АА)(– ts) = (AA)(—st) = (-А5)(Аг) 


Arg AS = (ATH As) Аа 
E. y= phen 


Consequently, e“‘e ^* = s 
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8.18 Prove that е? = 7. 


From the definition of matrix multiplication, 0" = 0 for n > 1. Hence, 


=> Lor-reX ov =1+0=1 


aco 


Supplementary Problems 


8.19 Determine the limit of each of the following sequences of matrices as k goes to =: 


12120-1 
rae k 3k*1 
d 2 k-1 
k+l RFI 

B к-к) 

cl. k+l 


8.20 The Bessel function of the first kind of order zero is defined as 
-$ Gu" 
Jo) рУ Toy 


For which matrices A is J, (A) well defined? 


8.21 Determine the conditions on matrix A that will make the following function well defined: 


2 1 
дА = У, FEI 


n 
8.22 Find (a) sin A and (b) e^ for A= [ i 
8.23 Find (a) cos A and (b) 3A" + 2A" for A= E za 


8.24 Епа (a) sin A and (b) cos A for the 3 х 3 zero matrix 
In Problems 8.25 through 8.31, find е^“ 


sus [$55] t» [1] sm [у] 


210 200 
4 5 
8.28 3 ES 8.29 f 2 | 8.30 f 2 | 


002 
=) 0 
8.31 021 
002 
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8.32 


8.33 


8.34 


8.35 


Find sin At for 


Solve X(t) = AX(r) + F(t); X(0) = С when 
_fo 1 
^-[8 4] 


Solve X(t) = AX(r) + F(r) when 


> 
0 
1 
ое 


Solve X(t) = AX(r) + F(t); X(0) = С when 
0 1 
^-[$ 3] 
{ 1 
Solve Problem 8.35 with C = | j 


Solve AX + ХВ = C when 


COON 


oono 
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Chapter 9 


Canonical Bases 


GENERALIZED EIGENVECTORS 


A vector X,, is a generalized (right) eigenvector of rank m for the square matrix A and associated 
eigenvector A if 


(A-AD"X,-0 but  (A-AD""X, #0 


(See Problem 9.1 through 9.4.) Right eigenvectors, as defined in Chapter 7, are generalized 
eigenvectors of rank 1. 


CHAINS 
A chain generated by a generalized eigenvector X,, of rank m associated with the eigenvalue A is 
a set of vectors {X,,, X, ,, X,, ;,..., X) defined recursively as 


X-(A-ADX,, (ј=т-1,т-2,...,1) (9.1) 


(See Problems 9.5 and 9.6.) A chain is a linearly independent set of generalized eigenvectors of 
descending rank. The number of vectors in the set is called the length of the chain. 


CANONICAL BASIS 


A canonical basis for ап пхп matrix A is a set of n linearly independent generalized 
eigenvectors composed entirely of chains. That is, if a generalized eigenvector of rank m appears in 
the basis, so too does the complete chain generated by that vector. 

The simplest canonical bases, when they exist, are those consisting solely of chains of length one 
(1.е., of linearly independent eigenvectors). Such bases always exist when the eigenvalues of a matrix 
are distinct. (See Problem 9.9.) The chains associated with an eigenvalue of multiplicity greater than 
one are determined with the following algorithm, which first establishes the number of generalized 
eigenvectors of each rank that will appear in a canonical basis and then provides a means for 
obtaining them: 

STEP 9.1: Denote the multiplicity of A as m, and determine the smallest positive integer p for 
which the rank of (A — AI)" equals n — m, where n denotes the number of rows (and 
columns) in A. 

STEP 9.2: For each integer k between | and p, inclusive, compute the eigenvalue rank number N, 
as 

N, = rank(A — А)! — rank(A — А)“ (9.2) 

Each N, is the number of generalized eigenvectors of rank k that will appear in the 
canonical basis. 

STEP 9.3: Determine a generalized eigenvector of rank p, and construct the chain generated by 
this vector. Each of these vectors is part of the canonical basis. 

STEP 9.4: Reduce each positive №, (k = 1,2,..., p) by 1. If all №, are zero, stop; the procedure is 
completed. If not, continue to Step. 9.5. 

STEP 9.5: Find the highest value of k for which N, is not zero, and determine a generalized 
eigenvector of that rank which is linearly independent of all previously determined 


& 
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generalized eigenvectors associated with A. Form the chain generated by this vector, 
and include it in the basis. Return to Step 9.4. 


(See Problems 9.10 through 9.13.) 


THE MINIMUM POLYNOMIAL 
The minimum polynomial m(A) for an n X n matrix A is the monic polynomial of least degree for 


which m(A) = 0. Designate the distinct eigenvalues of A as A,, А,,..., A, (1 5 s = n), and for each A, 
determine a p, as in Step 9.1 above. The minimum polynomial for A is then 
m(A) = (A — A)P(A = AP = (A= AP (9.3) 


(See Problems 9.14 and 9.15.) 


Solved Problems 


9.1 Show that X = [1, 0, 0]" is a generalized eigenvector of rank 2 corresponding to the eigenvalue 
А = 3 for the matrix 


0 02 
-10 -25 1 0 0 -36 
A-3I- 4 10 1 and (A-3I=/0 0 13 
0 0-1 00 1] 


For X = (1,0,0]", we have (A – 3DX = [-10, 4,0]" #0 and (A – 3X = 0, which implies that X is а 
generalized eigenvector of rank 2. 


9.2 Find a generalized eigenvector of rank 3 corresponding to the eigenvalue A = 7 for the matrix 


We seek a three-dimensional vector X, = [x,, x;, x;]" such that (A – 71)°X, = 0 and (A – 71)°X, # 


0. We have 
о о ofa] [o 
(A-71)X,=|0 0 o|x:|-|o 
o o 0х] Lo 
0 0 17] =f45 
and (А-1), = 0 0 0| х: |= 0 
0 0 0х) = 10 


The condition (А — 71)°X, = 0 is automatically satisfied; the condition (A — 71)°Х, # 0 is satisfied only if 
x, #0. Thus, x, and x, are arbitrary, whereas x, is constrained to be nonzero. A simple choice is 
‚ yielding X, = [0, 0, 1)”. 


x, 7x, 70. x, 


9.3 


9.4 
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Find a generalized eigenvector of rank 2 corresponding to the eigenvalue A — 4 for the matrix 


4 000 
d 1 510 
AP oT 50 
0 003 


We seek a four-dimensional vector X, = [х,, х. X}, x,]" such that (А — 41)°Х, = 0 and (A — 4DX, # 
0. We have 


ооо ox] [о 
10 оо 0|n]| [0 
(4-4 X7 o o o[x,|*|o 
ооо ilaj [x 
0 ох, 0 
A > к 1 1 0j% x, tx, +x, 
ang 679067], | р о ||, x 
0 0 о -ila -x 


To satisfy (A — 41)°X, =0, we must have x, =0. Then, to satisfy (A — 4I)X, #0, we must guarantee 
x,#—x,—x,. A simple choice is x, = 1, x, = x, = x, = 0. This gives us X, = (1,0.0,0]". 


Show that there is no generalized eigenvector of rank 3 corresponding to the eigenvalue A = 4 
for the matrix given in Problem 9.3. 


For such a vector X, = [х,, x3, ху, x,]" to exist, the conditions (A — 41)°X, = 0 and (A ~ 41)°X, #0 
must be satisfied. For the given matrix A, 


(А – 41)°X, =(0,0,0,-x,]7 — while (А – 41)?X, = (0.0.0, x,]’ 


To satisfy both conditions, x, must be zero and nonzero simultaneously, which is impossible. Therefore. 
A has no generalized eigenvector of rank 3 corresponding to A = 4. 


Determine the chain that is generated by the generalized eigenvector of rank 3 found in 
Problem 9.2. 


From Problem 9.2, we have X, = [0,0, 1]" corresponding to the eigenvalue A = 7. Furthermore, 


012 
A-712|0 0 1 


000 


0 1 210] [2 

калси 0 THER 

оо olli] LO 

от 2727 r1 

and еар 0 THEE 
o 0 olo) 10 


ТТТ 


It follows from (9.1) that 
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9.6 


9.7 


9.8 


9.9 


Determine the chain that is generated by the generalized eigenvector of rank 2 found in 
Problem 9.3. 


From Problem 9.3 we have X, =[1, 0, 0, 0]", corresponding to A = 4. Using (9.1), we write 


о о о оў: 0 
1 3 3 offo 1 
Xc-(4-43DX-7|.i „ли ыу o[o|*|-1 
o о о -illo 0 


The chain is (X,, X,) = ([1, 0, 0, 0]", [0, 1, —1,0]"). 


Show that if X,, is a generalized eigenvector of rank m for matrix А and eigenvalue A, then X, 
as defined by (9.1) is a generalized eigenvector of rank j corresponding to the same matrix 
and eigenvalue. 


Since X,, is a generalized eigenvector of rank m, 
(A-AD"X,-0 and  (A-AD"^"X,»*0 
It follows from Eq. (9.1) that 
X, = (А - ADX,,, = (A - AD" "X, 
Therefore (A - AD/X, = (A — AD'(A — AD""X,, = (A - AD"X,, = 0 
and (A — AIX, = (A — АГУ (A — А)" УХ, = (A - AD" "'X, 0 


which together imply that X, is a generalized eigenvector of rank j for A and A. 


Show that a chain is a linearly independent set of vectors. 


The proof is inductive on the length of the chain. For chains of length one, the generating 
generalized eigenvector X, must be an eigenvector, so X, 7 0. Therefore, the only solution to the vector 
equation c,X, = 0 is c, = 0, and the chain is independent. 

Assume that all chains containing exactly k — 1 vectors are linearly independent, and consider a 


chain consisting of the k-vector set (X,, X, ,,... , X,) for matrix A and eigenvalue A. We must show 
that the only solution to the vector equation 

eX, + с, Ха ++ eX =O a) 
is cy =c, 77:7 c, =0. Multiply (1) by (А — AD^*, and observe that for each X, (j= k - 1, k — 
2,...,1) in that equation, 


(A — AD" 'eX, = c(A - AD" А - АХ, = c (A -AD* "0-0 
because each X, is a generalized eigenvector of rank j (see Problem 9.7). What remains, then, is 
c,(A - А) xX, =0 (2) 

Since X, is a generalized eigenvector of rank k, (A — AI) 'X, #0, and it follows from (2) that c, =0. 
Equation (1) thus reduces to 

€, .,X,., +++ Ж, =O (3) 
But X, ,,...,X, is a chain of length К – 1, which we assumed to be linearly independent, so the 
constants c,_,,...,¢, in (3) must all be zero. Therefore, the only solution to (1) is c, 7c, , =" = 
с, = 0, from which it follows that the chain of length К is linearly independent. 


Determine a canonical basis for 


9.10 
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The eigenvalues for this matrix were found in Problem 7.1 to be A= 1 and A = —2. Since they are 
distinct, a canonical basis for A will consist of one eigenvector for each eigenvalue. Eigenvectors 
corresponding to А = 1 were determined in Problem 7.1 as x,[—5/2, 1] with x, arbitrary. We set x, =2 
to avoid fractions, and obtain the single eigenvector [—5, 2]". The eigenvectors associated with A = -2 
are x,[-1, 1]’ with x, again arbitrary. Selecting x, = 1 in this case, we obtain the single eigenvector 
[-1, 1]". A canonical basis for А is thus [-5,2]", [—1, 1]". 


Determine the number of generalized eigenvectors of each rank corresponding to A = 4 that 
will appear in a canonical basis for 


оороо ь 
ecoooan 
cocoate 
concoo 
canooo 
ree) 


For this 6 х6 matrix, the eigenvalue 4 has multiplicity five (while A = 7 has multiplicity one), so 
6, m=5, andn-m for Step 9.1. Now 


cococe 
oocooown 
сооооос 
осоюооо 
wooooo 


has rank 4, while 


(A - 4 = 


ocoooocococ 
ooooooc 
ооооооо 
ооооооо 
occooo-- 


has rank 2, and 


(A- 41} = 


eoccooc 

ооооое 

оооооо 

ооооое 

ооооое 
е 


has rank 1 = л — m. Therefore, р = 3. Using Step 9.2, we compute 
N, = rank(A – 4I)! – rank(A – 41)? =2-1=1 
М, = rank(A — 4I)' — rank(A – 4I)! = 4-2=2 
N, = rank(A — 4I)" — rank(A ~ 4I)' = rank(I) — rank(A – 4) -6- 4-2 
A canonical basis will contain one generalized eigenvector of rank 3, two generalized eigenvectors of 


rank 2, and two generalized eigenvectors of rank 1, all corresponding to A — 4. (It will also contain one 
generalized eigenvector corresponding to А = 7.) 
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9.11 Find a canonical basis for the matrix given in Problem 9.10. 


9.12 


We first find the vectors in the basis corresponding to A — 4, using the information obtained in the 
solution to Problem 9.10. There is one generalized eigenvector of rank p = 3, which we denote as 
X, = (х, хоху, X, ху, Xe]. We note that to have (A — 4I)'X, = 0, we must set x, = 0; and to have 
(А — 41)°X, #0, we must have x, »*0. A simple choice is X, = [0, 0, 1,0,0, 0]7, which generates as the 
rest of its chain 


02 100 ofo 1 
00-1000]0| |-1 

оо 00020]|1 0 
X-04-9X-|o 9 o020]|o| ^| о 
оо 0000]|0| | о 

оо ооо 3/0] Lo 
02 1 оооу 1] [2 

0 0 -1 0 0 0|-1 0 
000000] 0 0 

and X=(A-4D¥=19 0 o 0 2 off 0/7] о 
000000] 0 0 

оо о ооз о] Lo 


We next reduce each nonzero N, by 1, obtaining N, = 0, N, = 1, and N, = 1; thus, one generalized 
eigenvector of rank 2 and one generalized eigenvector of rank 1 associated with А = 4 remain to be 
found. We first seek another generalized eigenvector of rank 2, which we denote as Y,= 
Do Yas Ys» Yas Ys» Yel”. М we are to have (A — 4I) Y, = 0, then both y, and y, must be zero; and if 
(А - ADY, #0, then either у, or y, must be nonzero. A convenient choice that is linearly independent of 
X, X,, and X, is Y, = (0,0, 0, 0, 1,0]", which generates, as the remaining vector of its chain, 


0 2 тоо 010] [o 
0 0 -10 0 ollo] |o 
оо ооо 90]|o| |o 

Y,70743DY;7lo 9 оо 2 оо = 2 
оо 0000]1| јо 
оо ооо 310) LO 


Reducing each nonzero N, by 1 again, we obtain №, = 0, N, = 0, and №, = 0; so all the necessary 
basis vectors corresponding to A = 4 have been found. 

The eigenvector А = 7 has multiplicity one, so its contribution to a canonical basis is any eigenvector 
associated with it. One such eigenvector is Z, = [0,0,0,0,0, 1}7. A complete canonical basis for the 
6 x 6 matrix A is the set of six vectors (X,, X;, X,, ¥,, Y,, Z,} consisting of one chain of length three, 
one chain of length two, and one chain of length one. 


Find a canonical basis for 


0 1 
0 0 
a =} 
0 3 


ооо 
COUN 


Matrix A has order 4х 4 and eigenvalue A=3 with multiplicity four. Thus, n= 4, т = 4, and 
n-m-0.Here 
1 


4 


oooo 


2 
0 
0 
0 


has rank 2, while 
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0000 
2 [0000 
(^-3)7-l9 0 00 
0000 


has rank 0, so р = 2. Then 


ank(A — 31)' — rank(A – 31)? -2- 0-2 
ank(A ~ 31)? — rank(A – 31)' = rank(I) – rank(A ~ 31) -4- 2-2 


A canonical basis for A will contain two generalized eigenvectors of rank 2. We denote one of these 
as X, = x, X2, ху, X4]”. The condition (A —3I)'X, = 0 is satisfied by all four-dimensional vectors, so it 
places no constraints on X,. The requirement (A — 3I)X, » 0 is satisfied if either 


x,*0 or 2x, t x,*0 (1) 


A convenient choice, therefore, is X, = [0, 0,0, 1]7, which generates the remaining vector of its chain: 


020 үө 1 

ооо оо 0 
Xi2(73DX;7|o o o -1 {0} =] -1 
ооо oli 0 


We reduce №, and №, by 1, obtaining №, = №, = 1. Another generalized eigenvector of rank 2 for A = 3, 
linearly independent of X, and X, but satisfying (1), is Y, = [0, 1,0, 0]7. This vector generates 


ecco 


20 
оо 
00 
00 
Now №, and N, are reduced to zero; a canonical basis for A is thus (X;. X, Y}, Y,), comprised of two 


chains, both of length two. 


9.13 Determine a canonical basis for 


ooou- 
-NOVN 
NEOON 


The eigenvalues for this matrix are А = 3 with multiplicity four and A= 1 with multiplicity one. For 
A=3,n-m=5-4=1. Also, 


Uia 2 2 
-y r1 3 0 

A-32| 0 00 о 0 

0 00-1 1 

0 00 1-1 

has rank 3, while 

OW 2 * 2 

00-2-8 1 

(A-3)-]|00 0 0 0 

00 а 2-2 

00 02 2 


has rank 2, and 
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9.14 


9.15 


has rank 1. Thus, p = 3, and № -2- 1-7 1, N, 
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0 
0 
(A-3Dp'-|0 
0 
0 


-3=2. 


A generalized eigenvector of rank p =3 for ‚ 0]". which generates 


1 1 
-1 1 
X, =(A-31)X,=| 0 =|0 
0 0 
0 0. 
1 2 
-1 -2 
and X,-(A-3DX,-| 0 =| 0 
0 0 
0 0 


We now reduce the №, by 1 to obtain №, = №, = 0 and №, = 1. Thus, a canonical basis for А will contain 
generalized eigenvector of rank | for A = 3. This is an eigenvector, and it must be linearly independent of 
X,. X., and X,; to find it we solve (A — 31)Y, = 0 to obtain, as one possibility, Y, = [0. — 1, —7, 2, 2]". 

Since А =1 is an eigenvalue of multiplicity one, its contribution to a canonical basis is any 
eigenvector corresponding to it. One choice is Z, = [-3. 9,0, —4, 4]". A canonical basis for ће matrix A 
is, then, {X,,X,,X,.¥,,Z,}, comprised of one chain of length three and two chains of length one. 


Find the minimum polynomial for the matrix given in Problem 9.12. 


In Problem 9.12 we found that the matrix has the single distinct eigenvalue A = 3, with p = 2. Its 
minimum polynomial is then 


m(A)=(A-3P = А? - 6+9 


Find the minimum polynomial for the matrix given in Problem 9.13. 


From Problem 9.13 we know that the matrix has two distinct eigenvalues: A, = 3 with multiplicity 
four and p, = 3, and A, = 1 with multiplicity one and thus p, = 1. Then 


m(A) = (А - 3)(A - 1) = A - 10A* + 367 — 54А + 27 


90 CANONICAL BASES [CHAP. 9 


Supplementary Problems 


9.16 Determine which of the following are generalized eigenvectors of rank 3 corresponding to A = 1 for the 
matrix 


> 
[] 
EET 
ооо-о 
oono- 
0 


(а) [1,1,0,0,0]7 (5) [0,0,0,0,2]7 (о) [0,0,0,1,0]” 
(d) [1,1,0,2,0]7 (е) [0,0,0,0,0}7 (7) [1,1,1,1,1) 


9.17 Find the chain generated by X, = [0, 0, 0, 0, 1]’, a generalized eigenvector of rank 4 corresponding to 
А = 1 for the matrix given in Problem 9.16. 


9.18 Find a generalized eigenvector of rank 2 corresponding to А = 5 for the matrix 
510 
A=|0 5 1 
005 


9.19 Find the chain generated by X, = [0, 0, 1]", a generalized eigenvector of rank 3 corresponding to А = 5 
for the matrix given in Problem 9.18. 


9.20 Determine the lengths of the chains associated with an eigenvalue A that are included in a canonical basis 


it 
(a) №.= N =2 (b) Nj -1,N, =3 
(б N 2, *1,N 22 (4) №=2, №, =3, N, =3 


(e) №=1,№=2,№=5 (f) №=3, № =5 
(8) №22, №. =2, № =1 


In Problems 9.21 through 9.31, (а) determine the number of generalized eigenvectors of each rank that will 
form a canonical basis for the given matrix, and (6) find such a basis. 


9.21 The matrix in Problem 9.16. 9.22 The matrix in Problem 9.18. 


221 
9.23 [3 1] 9.24 E 2] 9.25 HE 
002 
2 3.2 2 14 4 тд 3 
92 |0 4 0| 927 | 1 2 1| эз | 2 4 2 
3 -31 32-204 -7 -3 -3 
3210 3110 eer. 
0300 0300 03 7 
9.29 9.30 931 оо 500 
-1 110 1110 
0113 0113 [e ap 
00 014 


9.32 Find the minimum polynomial for the matrix in 
(a) Problem 9.25. (b) Problem 9.26. (c) Problem 9.27. 
(d) Problem 9.28. (e) Problem 9.29. (f) Problem 9.30. 


Chapter 10 


Similarity 


SIMILAR MATRICES 
A matrix A is similar to a matrix B if there exists an invertible matrix S such that 
A=S"'BS (10.1) 
If A is similar to B, then B is also similar to A and both matrices must be of the same order and 
square, 


Property 10.1: Similar matrices have the same characteristic equation and, therefore, the same 
eigenvalues and the same trace. 


Property 10.2: If X is an eigenvector of A associated with eigenvalue A and (10.1) holds, then 
Y = SX is an eigenvector of B associated with the same eigenvalue. 


(See Problems 10.1 through 10.3 and 10.43.) 


MODAL MATRIX 


Associated with every square matrix A is a canonical basis (see Chapter 9). A modal matrix M 
for A is a matrix of the same order as A having as its columns all the vectors of a canonical basis for 
A. A canonical basis is a set of linearly independent vectors, so M has an inverse. 

A modal matrix M is not unique. To standardize M somewhat, we shall always assume it has 
been constructed as follows: 


(М1): All chains of length one precede all longer chains (if any exist). 


(M2): The vectors of each chain of length two or more are contiguous, with rank increasing from 
left to right. 


(See Problems 10.4 through 10.6.) 


JORDAN CANONICAL FORM 


A Jordan block is a square matrix whose diagonal elements are all equal, whose superdiagonal 
elements (those immediately above the main diagonal) all equal 1, and whose other elements are all 
zero. It has the form 


A Jordan block is completely determined by its order and the value of its diagonal elements. 
A matrix is in Jordan canonical form if it is a diagonal matrix or if it has one of the following two 
partitioned forms: 
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D a 


һ h 


0 Ug ИШИ 


where D denotes a diagonal matrix (whose diagonal elements need not be equal) and J, (i= 
152: es k) represents a Jordan block. Although the diagonal elements in any one Jordan block 
must be equal, different Jordan blocks within the Jordan canonical form may have different 
diagonals. (See Problem 10.7.) 


SIMILARITY AND JORDAN CANONICAL FORM 


Every square matrix A is similar to a matrix J in Jordan canonical form. If M is a modal matrix 
for A, then 


A-MIM' (10.2) 


Equation (10.2) has the form of (10.1) with S2 M'^*. 

The matrix J in (10.2) is uniquely determined by M. Each chain of length r appearing in M and 
corresponding to eigenvalue A generates an r X r Jordan block in J with A on the diagonal. The 
chains of length one (if they exist) give rise collectively to the diagonal submatrix of J; the diagonal 
elements of this submatrix are the eigenvalues associated with the chains of length one, in the same 
order as their corresponding eigenvectors in M. If M consists solely of (generalized) eigenvectors (of 
rank 1), then J is simply a diagonal matrix; if M contains no chains of length one, then J is a 
partitioned matrix of Jordan blocks. (See Problems 10.8 through 10.11.) 

The Jordan canonical form of a matrix is unique as regards the individual Jordan blocks it 
contains, as well as the diagonal elements associated with chains of length one, if they exist. 
However, the positions of Jordan blocks and diagonal elements are not unique. Each chain must 
appear as contiguous columns in a modal matrix, but there is no criterion for ordering the chains. 
Different orderings of entire chains will produce different permutations of the associated Jordan 
blocks in J, or of the diagonal elements of D. 

Jordan blocks are defined on occasion to have Os on the superdiagonal and 1s on the 
subdiagonal. Such forms are obtained easily by changing rule M2 so that rank decreases from left to 
right in M. 


FUNCTIONS OF MATRICES 


Functions of matrices are easily computed for matrices in Jordan canonical form. If J is the 
diagonal matrix 


A 0 0 
0 X 0 
Lag SN CNN 
0 0 A, 
KA) 0 0 
then f=]. 0. АА 4 0 (10.3) 


(See Problem 10.14.) If J is the r x r Jordan block 


CHAP. 10) SIMILARITY 93 


A 10 оо 
0A 1 0 0 
ed б. a 
000 
000 
КА) FO fA |. f7?0). fO) 
0! 1! 2! (r-2)! (r-1) 
o FA LW |, ff 70) ft ?Q) 
then =| ...... Е ee o NE (10.4) 
(A "(А 
К) 
0 0 0 0 "a 


where all derivatives are taken with respect to A. (See Problem 10.15.) If J is a partitioned matrix of 
Jordan blocks and (perhaps) a diagonal matrix such that 


D 
Jj, 0 
J= 
0 
» 
f(D) 
then m- 0 0 
0 fan 


where f(D) and f(J;) (i 7 1,...,k) are defined by (10.3) and (10.4), respectively. (See Problem 
10.16.) If A is similar to the matrix J in Jordan canonical form, then 


ЖА) = Mfa)M " (10.5) 


(See Problems 10.17 through 10.19.) This formula is computationally efficient only when M is known 
or easily found; otherwise the procedure given in Chapter 8 is preferred. Since f(J) is upper 
triangular, (10.5) and Property 10.1 imply: 


Property 10.3: If A is an eigenvalue of A, the f(A) is an eigenvalue of f(A). 
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Solved Problems 
10.1 Determine whether 
„[2 0 —À = [2 0 
A É 2 is similar to B= 02 
The matrices are similar if and only if there exists a matrix S such that A = S~'BS or, equivalently, 
such that 
SA=BS a) 
[ab 
Set s= [2 d 


Then (1) becomes 


[e alls 2]- (0 olle 4] 
2a*3b 2b] [2a 2b 
n [2:3 2d -[32 24 


The solution to this matrix equality is b = d = 0, with a and c arbitrary. For (1) to be valid, S must thus 
have the form 
_ [а 0 
m k o) 


which is nonsingular for any choice of a and c. Thus, there is no invertible matrix which satisfies (1), and 
the matrices A and B are not similar. Observe that A and B have the same eigenvalues, so matrices may 
have the same eigenvalues and not be similar. 


10.2 Determine whether 
3 1:1 211 
A=|2 2 4 is similar to B=|3 0 1 
1141 421 


The trace of A is 3+ 2+ 1 = 6, while that of B is 2 + 0 + 1 =3. Since the traces аге not equal, A and 
B must have different sets of eigenvalues and, therefore, cannot be similar. 


10.3 Prove that similar matrices have the same characteristic polynomial. 
If A and B are similar, there exists a matrix S such that A = S 'BS. Therefore, 
JA - al] = |8 'BS - AS^'S|] = |S~'(B— ADS] 


1 


= 18718 - al}}s| = isi 


IB — ailis} = IB — АЦ 


10.4 Construct a modal matrix for 


Cooney 
оон 
ANOUN 
NEOON 
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A canonical basis for A was found in Problem 9.13. It consists of one chain of length three, 
X,-(0,0,1,0,07 — X,-(1,1,0,0,0 Х, = (2, -2,0,0,0]" 
and two chains of length one, 


Y, =(0,-1,-7,2,2]7 and  Z,-(-3,9,0,-4,4]" 


0-3 210 
-1 9-210 
Thus, М = (ү,,2,,Х,,Х,,Х,]=[-7 0 001 
2-4 000 
2 4 00 0 


A second modal matrix may be obtained by interchanging the first two columns of M. 


10.5 Construct a modal matrix for 


42 1000 
04-1000 
л-|00 4000 
00 0420 
00 0040 
00 0007 


A canonical basis for this matrix was determined in Problem 9.11 to consist of one chain of length 
three, 


X,-[0,0,1,0,0,07 — X,=(1,-1,0,0,0,0]7 X, =(-2,0,0,0,0,0)” 
one chain of length two, 
Y, =(0,0,0,0,1,0]7 ү, -(0,0,0,2,0, 0]" 
and one chain of length one, 
Z,  (0,0,0,0,0,0 1]" 


A modal matrix for A is either M=(Z,, Y,, Y}, X, X2, X;] or 


0-2 1000 
0 0-1000 
0.0 0100 
M-s[Z,X.XoX.Y,Yl7|9 9 0020 
0.0 0001 
1.0 0000 
10.6 Construct a modal matrix for 
522 
А=|3 6 3 
669 


A set of three linearly independent eigenvectors for A was determined in Problem 7.4 to consist of 
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[-1.1,0]', [7 1.0, 1]^. and [2.3. 6|". Since these three vectors form a full complement of generalized 
eigenvectors of rank 1. they are a canonical basis for A. A modal matrix for A is then 


=f o1 2 
M= 1 0 3 
0 1 6 


Any permutation of the columns of M will produce another equally acceptable modal matrix. 


Determine which of the following matrices are in Jordan canonical form: 


000000 
3000 18576 010000 
0300 B _loo1100 
ASKo o q o4 P-7[D 8 ур кеа 0 
0002 25704 000011 

000001 
All three matrices are in Jordan canonical form: A, because it is a diagonal matrix; B, because it is 
in the form 
310 
J А 41 
s-[5 with J |J 3 | ad J 
0 у ax ar НА 
and C, because it is in the form 
р 
0 0 0 $c 
c=| J, with D R | аа se M | 
Ü x 


Find a matrix in Jordan canonical form that is similar to the matrix A of Problem 10.6. 
Using the results of Problem 10.6, we note that the columns of M are all eigenvectors (chains of 
length one) corresponding, respectively, to the eigenvalues 3, 3, and 14 (as found in Problem 7.2). Thus, 


A is similar to the diagonal matrix 
30 0 
J=|0 3 0 
0 0 14 


To see that it is. note that 


-1 -1 2] [3 0 oj-321 ви -3u 
MJIM'-| ! 0 3/0 3 0j|-6n -etn 5/11 
o 1610 o m)l tat dH ип 


Find a matrix J in Jordan canonical form that is similar to the matrix A of Problem 10.4. 


In Problem 10.4 we found that M = [Y,, Z,. X,. X.. X,]. The two generalized eigenvectors of rank 
1. Y, and Z,. correspond to the eigenvalues 3 and 1, respectively. and generate the diagonal submatrix 


of J: 
ET 
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The chain of length three, (X,, X,, X;}, corresponds to the eigenvalue A = 3, so it generates the Jordan 


block 5 8.15 
EH 
um 003 
A is thus similar to 
30000 
01000 
i [D S]-joosra 
е 00031 
00003 


Find a matrix J in Jordan canonical form that is similar to the matrix A of Problem 10.5. 


In Problem 10.5. we found that М = [Z,, X, , X,. X, ¥,. Y.]. The single generalized eigenvector of 
rank 1, Z,, corresponds to the eigenvalue 7 and generates the 1 х 1 diagonal submatrix of J comprised of 
this eigenvalue. The chain of length three, (X,, X;. X,), corresponds to the eigenvalue 4 and generates 


the Jordan block 
410 

з= |041 

004 


The chain of length two, (Y,, ¥,}, also corresponds to the eigenvalue 4 and generates the Jordan block 


ІА 1 
04 
Thus A is similar to "TT 
m 0] {о 41 000 
ja А a 04100 
000400 
0 3) |ооооз 1 
0000 0 4, 


to a matrix A whose characteristic 
1 and 


Find a matrix J in Jordan canonical form that is similar 
equation is (А -2)'(A – 3) =0, and that has eigenvalue rank numbers №, = №, 
М, = 3 associated with eigenvalue A —2, and N, = 2 and №, = 3 associated with A 


A must be 10 x 10 matrix, and so too is J. A canonical basis for A will contain one chain of length 
three and two chains of length one corresponding to A = 2. Denote these vectors, respectively, as X,(2). 
X,(2), X,(2). Y,(2). Z, (2), where the integer in parenthesis denotes the associated eigenvalue. This 
same canonical basis must also contain two chains of length two and one chain of length one 
corresponding to А = 3. Denote these vectors, respectively, as 0.(3). U,(3). V.(3). V,(3). W,(3). We use 
this canonical basis to construct the modal matrix 


M = [Y,(2). Z,(2). W,(3). U,(3), U,(3), X,(2). X.(2). Х,(2). V,(3). V.(3)] 
which determines the matrix in Jordan canonical form 


200 
020 
003 


оо 
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10.12 Verify (10.2) for a modal matrix consisting solely of generalized eigenvectors of rank 1. 


, E,, where each E, (i= 1, 2,..., п) is an eigenvector of A. 
..,A, of A need not be distinct. Now define 


Denote the columns of M as E,, E,, . 
Thus, AE, = A,E,. The eigenvalues A,, Az, 


and note that 
AM = А[Е,,Е,,...,Е,]= (AE, AE,,..., AE,] 
=[A\E,, A2E,.-- , А„Е„] = [E E, . - - , E„]J = MJ 


from which (10.2) immediately follows. 


10.13 Verify (10.2) for a modal matrix consisting of a single chain of length r. 


Denote the columns of M as X,,X,,...,X,, where each X, (i—1,2,...,7) is a generalized 
eigenvector of rank i for A and all X, correspond to the same eigenvalue A. Now 


X,_, = (A- АХ, = AX, - AX, 
X A – АХ, 


X, = (А - А)Х, = AX, - AX, 
X, = (A - ADX, = AX, - АХ, 
which may be rewritten in reverse order as. 
AX, = AX, +X 
AX, = AX, +X, 


In addition, since X, is an eigenvector, AX, = AX,. 
Define J to be the r X г Jordan block with A on the diagonal. Then 


AM = A[X, X, X, ..., X,] = [AX,, AX,,AX,,...,AX,] 
=[AX,, AX, +X,, AX, X4. .., AX +X, a] 
T[X,X, X,,..., XJ - MJ 


and (10.2) follows immediately. 


10.14 Calculate sin J for the diagonal matrix 


Here ДА) = sin A, so it follows from (10.3) that 


sin3 0 0 0.141120 0 0 
з) 0 si3 0 j= 0 0.141120 0 


0 0  sinl4 0 0 0.990607 


CHAP. 10} SIMILARITY 


10.15 Calculate sin J for the Jordan block 


ooow 
Done 
оон о 
о-оо 


Here f(A) = sin А, f'(A) = cos А, f'(A) = -sin А, and f"(A) = —cos A, so f(2) = sin2, f'(2) = cos2, 


О) = -sin 2, and f"(2) = —cos 2. 


sin2 cos2 -sin2 
1 1 2 
o m зг 
fn- i 
o о #2 
1 
0 0 0 


10.16 Calculate cos J for 


This matrix is in Jordan canonical foi 


0 


»-[) 


It follows from (10.4) that 


—cos2 
6 
—sin2| [0.909297 —0.416147 -0.454649 0.0693578 
2 ы 0 0.909297 —0.416147 —0.454649 
cos 2 0 0 0.909297 — —0.416147 
1 0 0 0 0.909297 
1 
0000000 
0000000 
0001000 
J=|0 000000 
0000010 
0000001 
0000000 
rm, with all eigenvalues equal to zero and with the blocks 


010 
e|. ae [S el sfo o 3] 
000 


Since f(A) = cos A, we have f(0) = cos 0, /'(0) = —sin 0, and f'(0) = —cos 0. Therefore, 


_[cos0 НАЕТ 
cosD=| 9 cosd [i 
[cos0 -sinO 
.| o a qno 
eod, б зо: |*[0-1 
or 
[cos0 -sin0 -cos0 
07 и 2! 
d 1 0 -1/2 
oss| o S ORE o°] 
à : 00 1 
cos 0 
[o о 5% 
100000 0 
010000 0 
[ cosD 0 001000 0 
and cosJ= cos J, 2.000100 о 
| 0 со„] Jo 0001 0 ~1/2 
000001 0 
000000 1 
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10.17 Calculate sin A for the matrix given in Problem 10.6. 
Using the results of Problems 10.6, 10.8, and 10.14 along with (10.2), we have 


-1 -1 2][0.141120 0 0 -3n 8/11 -3/n 
зїп А = Msin)M '-| 1 03 0 0.141120 0 -6/11 -6/1 5/11 


0 1 6 0 0 0.990607 1/11 111 1/11 


0.231678 0.372798 0.231678 


[ozmen 0.154452 nai | 
0.463357 0.463357 0.604477 


10.18 Calculate e^' for 


и 0 
We set B=Ar=|0 2t г 


(see Chapter 8) and calculate e”. Even though A is a Jordan block, B is not because it no longer has 1s 
on the superdiagonal. We find that modal matrix for B is 


Poo 
M-j0 7 0 
001 


so B is similar to the matrix in Jordan canonical form 
21 0 
J=M'BM=|0 27 1 
0 0 2 
We have f(A) = (А) = (А) = e", so 
Pd e" * 


mes ees 
0 0 e 


1 + 72 
and еме = В) = МАЈ)М '=e"10 1 г 
00 


10.19 Find е^ for 


rx р il 
We set Ы 0 
and compute e”. The eigenvalues for В аге the complex conjugates it and — iz, so B is similar to 
I 0 
0 -ir 


A modal matrix for B, consisting of two generalized eigenvectors of rank 1, is 


J 
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T 1 ie" 0 1/2 -im 
Mw" [i “lip 125 12 


Thus, 
dre" mie tie 
18 2 cost sin ] 
i-i dece] —sint cost 
2 2 


(Compare with Problem 8.7.) 


Supplementary Problems 


10.20 Determine which of the following pairs of matrices are similar matrices: 


23 ST 
e» $3 +в 5i 
21 20 
e [oa] am [032] 
r2 89 221 
(c) 456 and 112 
789 122 
In Problems 10.21 through 10.25, find a modal matrix associated with the given matrix. 
-5 -3 13] , 
na [5 1] юз [15] 
(Hint; See Problem 9.23.) (Hint: See Problem 9.24.) 
21:0: 7 2.0.7 
10.23 |0 2 0 10.24 |0 4 0 
002 3 23 1 
(Hint: See Problem 9.25.) (Hint: See Problem 9.26.) 
3210 
0300 
10:25; =) Ж. 
0113 


(Hint: See Problem 9.29.) 


10.26 Each of the following is a complete set of eigenvalue rank numbers for a matrix whose only eigenvalue is 
А = 2. Find, in each case, a matrix in Jordan canonical form which is similar to that matrix. 


(a) N,=N,=2 (b) №=1, №, =3 
(с) Ny=N,=1,N,=2 (d) N,=2, N,=3, N, =3 
(e) N,=1,N,=2,N,=5 (f) N,=3,N,=5 


In Problems 10.27 through 10.33, find a matrix in Jordan canonical form that is similar to the given matrix. 


10.27 The matrix in Problem 10.21. 10.28 The matrix in Problem 10.22. 
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10.29 The matrix in Problem 10.23. 10.30 The matrix in Problem 10.24. 


10.31 The matrix in Problem 10.25. 


3110 51 200 
0300 05-100 

1932 |_1 1 1 0 1033 |0 0 500 
0113 00 032 

00 014 


(Hint: See Problem 9.30.) 


10.34 Find cos A for the following matrices: 


200 210 

(a) А=|0 2 0| (6) A-|0 2 1 

002 002 
200000 
2100 020000 
.jo200 _jo02100 
© A=lo 021) @ A*lo 00210 
0002 000021 
000002 


10.35 Find e^ for the matrix A given in Problem 10.21. 
10.36 Find e^ for the matrix A given in Problem 10.23. 
10.37 Find А”! for the matrix A given in Problem 10.23. 


In Problems 10.38 through 10.41, determine e^. 


was a=[2 1] юз a=[_5 $] 


02 4 - 

200 -110 
10.40 A= [ 2 | 10.44 A-| 0 2 | 

002 002 


10.42 Prove that if A is similar to B, then B is similar to A. 


10.43 Prove that if (10.1) is valid for A and B and X is an eigenvector for A corresponding to eigenvalue A, 
then Y = SX is an eigenvector for B also corresponding to A. 


Chapter 11 


Inner Products 


COMPLEX CONJUGATES 


The complex conjugate of a scalar z = a + ib (where a and b are real) is Z = a — ib; the complex 
conjugate of a matrix A is the matrix A whose elements are the complex conjugates of the elements of 
A. The following properties are valid for scalars x and y and matrices A and B: 


(C): ¥=x; and À =A. 

(C2): х is real if and only if x = x; and A is a real matrix if and only if A= A. 
(C3) x isa real scalar; and A + А is a real matrix. 

(C4): зу = (X)( y); and AB = (А)(В) if the latter product is defined. 

(C5): (x y)-X-* y; and (A +В) = А +В if the latter sum is defined. 


(Сб): xi = |x|? is always real and positive, except that xx = 0 when x = 0. 


THE INNER PRODUCT 


Let W denote a nonsingular n X n matrix. The inner product of n-dimensional column vectors X 
and Y with respect to W, denoted by (X, Y)w, is the dot product (see Chapter 1) 


(X, Y), = (WX) (WY) (11.1) 
If W=I, then the subscript in (11.1) is dropped, and the inner product 
(X,Y) =X-¥ (11.2) 


is called the Euclidean inner product. If X and Y are also real, then the Euclidean inner product 
reduces to the dot product of the two vectors. (See Problems 11.1 through 11.5.) 


PROPERTIES OF INNER PRODUCTS 

Property 11.1: (X, X), is real and positive if X #0. 

Property 11.2: (X, X),, — 0 if and only if X = 0. 

Property 11.3: (X, Y)w= (Y, X) v. 

Property 11.4: (cX, Y), = c(X, Y),, and (X, сҮ) = С(Х, Ү) for any scalar c, real or complex. 
Property 11.5: (X +Y, 2), = (X, 2), + (Y, Z) y and (X, Y +7), = (X, Y), + (X, Z)w- 
Property 11.6 (Schwarz inequality): |(X, Y) ^ s (X, X) (Y, Y)... 

(See Problems 11.9, 11.10, 11.12, 11.28, and 11.29.) 


ORTHOGONALITY 


Two vectors are orthogonal if their inner product is zero. Since different matrices W in (11.1) 
generate different inner products, two vectors may be orthogonal under one inner product and not 
orthogonal under another inner product. (See Problem 11.5.) Orthogonality reduces to the geometric 
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concept of perpendicularity under the Euclidean inner product when the vectors are real and 
restricted to two or three dimensions. 

A set of vectors is orthogonal if each vector in the set is orthogonal to every other vector in that 
set. Such a set is linearly independent when the vectors are all nonzero. (See Problem 11.27.) 


GRAM-SCHMIDT ORTHOGONALIZATION 


Every finite set of linearly independent vectors {X,,X,,...,X,,} has associated with it an 
orthogonal set of nonzero vectors {Q,,Q,,...,Q,} with respect to a specified inner product, such 
that each vector Q, (/= 1. 2,..., п) is a linear combination of X, through X,_,. The following 
algorithm for producing the vectors Q, is called the Gram-Schmidt orthogonalization process. 
STEP 11.1: Set 


1 
Q, = ———— 
М.Х) 
STEP 11.2: If j= n, stop; the algorithm is complete. Otherwise, increase j by 1 and continue. 
STEP 11.3: Calculate 


X, and ј = 


Y,-X,- У (X,,Q)wQ 


STEP 11.4: Set 


STEP 11.5: Return to Step 11.2. 


(See Problems 11.6 through 11.8) A modification of this algorithm, which is less susceptible to 
roundoff error, is presented in Chapter 20. 

Besides producing orthogonal vectors, the Gram-Schmidt process generates vectors having the 
property that the inner product of each vector with itself is unity. This property is discussed further in 
the next chapter. 


Solved Problems 


1 4 1. L0 
X-|2 Y=|5 же 11 
3 6 | | 


All elements are real, so the conjugate notation in (//./) can be suppressed. Thus. 


11.1 Calculate (X, Y), if 


зүр 
ЕЕЕ 122 
4 10. 


11.2 Calculate the Euclidean inner product for the vectors given in Problem 11.1. 


Since both vectors are real, the Euclidean inner product is the dot product of those veciors: 
(X.Y) = 1(4) + 2(5) + 3(6) = 32. 
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1.3 Calculate (X, X), if 
fie. ШИ 
x-[ i | аш wf, s d 


EJ — 1 
Hee wx-[; 745] © МХ а] 


and (X. X), = (WX) - (WX) = как (C76) + (7 + 13)(7 ~ i3) = 59 


-i2 i2 ] 2-5 
X2|3-i2 к= + W=|4 5 6 
i 7 T 8 9 


W is a singular matrix, so the inner product (X, Y), is not defined, even though the matrix 
operations on the right side of Eq. (11.1) can be performed. When the matrix W is singular. it is always 
possible to find a nonzero vector Z (in this case, Z = [1. —2, 1]' will do) for which (Z, Z),, = 0. thereby 
violating Property 11.2. 


11.4 Calculate (X, Y), if 


31.5 Calculate the inner product (X, Y) , for the vectors given in Problem 11.4 when (a) W = I and 
(b) W is as given in Problem 11.1. 


(a) The Euclidean inner product is 


-i2] f -i2 
(X,Y) = X-Y- ЕЕЕ 
i 7 
(b) With W as given in Problem 11.1, we have 
WX = [3- i4, and = WY=(2+13, 9+1. 7+ 2)’ 


so (X. Y), = (WX) - (WY) = (3- i(2 - i3) + (3 – i(9- i) + (-0(7 - i2) = 18 136 


Thus, X and Y are orthogonal under the Euclidean inner product but not orthogonal under the 
inner product in part b. 


11.6 Use the Gram-Schmidt orthogonalization process with the Euclidean inner product to 
construct an orthogonal set of vectors associated with (X, X,) when 


zh 2 P a, 4 

x-[.24] and х= [+] 

These two vectors are linearly independent, so Steps 11.1 through 11.5 may be used to find 
(X, X,) »X:X,79 


[ә] [ауз 


Also. v,-x.- .900,-[ 1] - (i^ 


i 215 [662 i7)/9 
B qei2y3]| | (K+ i2)/9 


so (Y, Y.) = Y,-Y, = 153/81 
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EC 6+ i7)/9] _[(—6 + i7)/V153 
and О, = 57733787 | (8+ i2)/9 || (8+ i2)/V 153 


The orthogonal set is (Q,, Q.). 


Use the Gram-Schmidt orthogonalization process with the Euclidean inner product to 
construct an orthogonal set of vectors associated with (X,, X,, X,, X,) when 


1 1 

0 1 
х= X=) Х,= |0 Же 
1 1 


0 1 
1 t 
1 1 
1 0. 


These vectors can be shown to be linearly independent (see Chapter 6). Using Steps 11.1 through 
11.5, we find 


(X,,X,) 2 X,-X, =3 

and Q,= TAY. =. ПУЗ, 1/V3, V31" 
Then Y, =X, -(X,,0)0, =X,- A. 

=([1, -2/3, 1/3, 1/3]" 
so (Ya, Yj) = Y, Y, = 15/9 
and Q,- V97ISY, = (3/V 15, -2/V15, 1/V15, 1/V T5)" 
Also, Y, =X, - (X,, Q,)Q, - (Х,.0,)0, 

=X,- 20. - 239. = (3/5, 3/5, —4/5, 1/5]” 

so (Y, Y) =Y,- Y, = 35/25 
апі 0, = У25735Ү, = [3/V35, 3/V35, —4/\/35, 1/V'35]" 


Lastly, Y,7 X, - (X,.0)Q, - (Х,.0,)0, - (Xa, Qs) Qs 
2 


" E ge 5. 
=a mL vs 


= [3/7, 3/7, 317, -6/7)" 


so (Ya, ¥,) = Y,- Y, = 63/49 

and О, = V49763Y, =[3/V63, 3/V63, 3/V63, —6/V63]" 
-2[UV7, 1/7, 1/V7, 2%] 

The orthogonal set is (Q,, Qz, Qs, Q.). 


Use the Gram-Schmidt orthogonalization process to construct an orthogonal set of vectors 
associated with the set {X,,X,,X,} from Problem 11.7 and with 
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i sp Bod 
0 1 23 
Wale od dà 
2-1-20 


Using Steps 11.1 through 11.5, we calculate 
WX, = [1,6.4, -3]" 
(X, X). = (WX,) + (WX,) = 62 
‚={0, 1/V62, 1/V82, 1/V62]" 
WQ, - [1/V62, 6/62, 4/\/62, -3/v/62]" 


and WX, -[3, 5, 5, 0]" 
so (Xz, Q,)w = (WX): (WQ, ) = 53/V 62 
and Y,7X,- (Xs. Q) 7 X. - 2250, 
=[1, 53/62, 9/62, 9/62]" 
Now WY, = [133/62, —8/62, 98/62, 159/62]" 
so (Y, Yz) = (WY,)* (WY,) = 52,638/(62) 
and Q,7 VER Y= ие. -53, 9, 9]7 
Lastly, wo, = VEU. 8, 98, 159]" 
and WX, = [0, 4, 5, 1]" 
so (XQ) w= (WX,)* (WQ,) = 41V62 
and (X4, Qs) w = (WX,) + (WQ,) = 617///52,638 
giving Y, =X, - (X,,0,)4Q, - (X3, Q2)wQ: 
ех, 059-0389 
= x35 (7,192, 25,265, —20,181, 6,138]" 
Now зҮ, = зб [758,435, 3,317, 31,744, 29.481]" 
so (Y, Ys) w= (WY,)  (WY,) = CU 
and Q= аат (7.192, 25, 265, -20,181, 6,138)” 
The orthogonal set is (Q,, Q2, Q;). 


11.9 Prove Property 11.3: (X, Y), = (Y, 
(X, Y) w = (WX) - (WY) = (WY)- (WX) = (WY) (WX) = (Y. X), 


we 
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11.10 Prove Property 11.4. 
(cX, Y) w = (WeX) + (WY) = (c(WX)}  (WY) = c((WX)- (WY)) = с(Х, Y), 


and (X, CY) w = (WX) + (WEY) = (WX) - {c(WY)} = (WX) - ((WY)) 
= (МХ) > (WY)} = 2(Х, Y), 


11.11. Prove that (0, Y), = 0 for any Y of appropriate dimension. 
(0, Y), = (00, Y), =0(0, Y), =0 


because the inner product is a scalar, and zero times any scalar is zero. 


11.12 Prove the Schwarz inequality. 


When X = 0, both sides of the inequality are zero (see Problem 11.11), and the inequality is 
satisfied. If X #0, then (X, X) #0 (Property 11.2), and for any vectors X and Y and any scalar c, we 
have 


0s (cX - Y, cX- Y), (Property 11.1) 
= (eX, eX) w – (cX, ¥)w- (Y, eX), * (Y. Y), (Property 11.5) 
= сё(Х,Х) — (Xs Y), - ё(Ү,Х)„ + (Y, Y)w (Property 11.4) 


Setting c = (X, Y) w/(X, X)w and noting that (X, X)w = (X, X), by Property 11.1, we cancel the first 
two terms on the right side of the last equality and obtain 


(X. ү), (У.Х), (X. ¥) w(K. ү), 


05 aah "(XT GK MS Ww (Property 11.3) 
» " 
т Ed *OSÁ. (Property C6) 
Их. Yal 
Thus, AX =(Y,Y) 


from which the Schwarz inequality immediately follows. 


Supplementary Problems 


11.13 Calculate (a) (X, Y), (b) (X, 2), (c) (Y, Z), (4) (X. ¥),, (e) (X,Z) 4. (f) (X, Y),. and (8) (X. 2), 


when 
1 2 1 1 23 100 
aH v] 2-|-1| ШЕ =, | z 2 o] 
1 0 H 1 21 00 3 
11.14 Calculate (а) (X, Y), (b) (Y. X), (c) (X, Z), (d) (Y Z), (e) (X, Y). and (f) (X, Z) , when 


x-fi) vp] 5:3) vbi 


In Problems 11.15 through 11.24, use the Gram-Schmidt orthogonalization algorithm with the Euclidean 
inner product to produce an orthogonal set of vectors from the given set. 
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11.25 
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[1] 
x =|} X,= 


BH 


Ono-~ 


Use the Gram-Schmidt orthogonalization algorithm to construct an orthogonal set of vectors associated 
with 


x] m xe] om wel} 3) 


Use the Gram-Schmidt orthogonalization algorithm to construct an orthogonal set of vectors associated 
with (X,, X,, X), where 


TEE PEU 


Prove that an orthogonal set of nonzero vectors is linearly independent. 
Prove Property 11.1. 


Prove Property 11.2. 


Chapter 12 


Norms 


VECTOR NORMS 


A norm for an arbitrary finite-dimensional vector X, denoted ||X||, is a real-valued function 
satisfying the following four conditions for all vectors X and Y of the same dimension: 


(ND): |IX|| =0. 

(М2): ||X|| =0 if and only if X = 0. 

(№3): lIcXI| = {cl [|X| for any scalar c. 

(N4) (Triangle inequality): |X + Y|| = 11х11 + || Vl]. 


A vector norm is a measure of the length or magnitude of a vector. Just as there are various bases for 
measuring scalar length—such as feet and meters—there are alternative norms for measuring the 
magnitude of a vector. Some of the more common vector norms for X = [x,, xz,- ..  x,]" are: 


e The inner-product-generated norm: ||X||w = V(X, X) w 

e The Euclidean (or 1,) norm: |X||, = V(X. X) 

© Thel, norm: ||Х||, = |х\| + [xal ++°° 1x] 

* Thel, norm: |X||. = тах (|х\|,|х,|,...,|х„|) 

e The 1, norm (p 21): ||Х||„ = (|х||” + Leal? Ix |^)" 

The Euclidean norm is the most popular, and it is a special case of the inner-product norm when 
М = E; the Euclidean norm and the /, norm are special cases of the /, norm for p=2 and p = 1, 


respectively. (See Problems 12.1 through 12.3.) Finally, in the limit as p>, the /, norm yields the 
l, norm. 


NORMALIZED VECTORS AND DISTANCE 


A unit vector is a vector having norm equal to unity. A nonzero vector is normalized when it is 
multiplied by the reciprocal of its norm; consequently, normalized vectors are unit vectors. A set of 
vectors is orthonormal if the set is orthogonal and if each vector in the set is a unit vector. 

The distance between two vectors X and Y is ||X — Ү||. Its value, as well as the designation of a 
vector as a unit vector, depends on the particular norm selected. (See Problems 12.4 and 12.5.) 


MATRIX NORMS 


A norm for a square matrix A, denoted ||A||, is a real-valued function satisfying the following 
conditions for all п X n matrices A and B: 


(м1): |A|| z 0. 

(М2): ||A|| = 0 if and only if A = 0. 

(М3): ||сА|| = |cl|[A]| for any scalar с. 

(M4) (Triangle inequality): |А + В| = ||A]|  ||BI| 

(М5) (Consistency condition): |АВ|| = ||A||||BI| 
110 
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Because of the added consistency condition (М5), not all vector norms can be extended to become 
matrix norms. (See Problem 12.6.) Two that can be extended are the /, norm (see Problem 12.7) and 
the Euclidean norm. For the л x л matrix A = [а„], the Euclidean norm becomes 


non m 
e The Frobenius (or Euclidean) norm: |А||; = (X x la!) 
m 


INDUCED NORMS 
Each vector norm induces (or generates) the matrix norm 
ПАП = max (АХ) агл) 


оп an arbitrary n X n matrix A where the maximum is taken over all n-dimensional vectors X having 
vector norm equal to unity. Some induced norms for A = [4,)] are: 


e The L, norm (induced by the l, norm): 


lal, = max, (2 lu) 


112... 


which is the largest column sum of absolute value. 


e The L, norm (induced by the 1„ norm): 
IAM. mex. (3: tay! 


which is the largest row sum of absolute values. 
e The spectral norm (induced by the Euclidean norm): 

||Al[; = max(V À: A is an eigenvalue of A'A) 
(See Problems 12.8 through 12.12 and Problem 15.12.) 


COMPATIBILITY 
A vector norm is compatible with a matrix norm if 
ТАУ s ТАУ (12.2) 
for every n X n matrix А and every n-dimensional vector Y. Induced norms are always compatible 
with the vector norms that generated them, and in those cases there always exists at least one vector 
Y for which (12.2) is an equality. (See Problem 12.13.) Compatibility is not restricted to induced 


norms; the Frobenius norm, for example, is compatible with the Euclidean vector norm even though 
the former is not induced by the latter. (See Problems 12.15 and 12.16.) 


SPECTRAL RADIUS 

The spectral radius of a square matrix A, denoted by с(А), is the largest absolute value of any 
eigenvalue of A. That is, с(А) = max (||: А is an eigenvalue of A). If A is any eigenvalue of a matrix 
A, then |A| = (A), and there is at least one eigenvalue for which this inequality is an equality. For 
any matrix norm, 


o(a) s АІ (12.3) 
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Inequality (12.3) provides bounds on the eigenvalues of a matrix. (See Problems 12.17 and 12.18.) 
An equivalent expression for the spectral radius is 


о(А) = lim ||A"||"" (12.4) 


Solved Problems 


12.1 Determine ||X||, and ||Ү||„ for 


For the given vectors, we have 


lIXIL, = У(Х. ХУ = V(OWX) (WX) = V30) + 505) + 4(4) = 50 


and Ү||,, = ҮСҮ. Y) w = МҮ) (WY) = V9(9) + TIC) + 10010) = У302 


12.2 Find (a) ||Х||›, (b) |IXII,. (с) ХІІ... and (4) ХІІ. for the vector X of Problem 12.1. 


(a) IX], = УХХ = VIC) + 22) + 33) = У. 
(b) IXI, = J1] + 12] + |3| = 6. 


(c) [XI]. = max({1}, 121, 3) = 3. 
(а) ХІІ. = (111° + [21 + 131°)" * = 76) * = 3.077. 


12.3 Find (а) ||X]]w, (b) ИХ. (© III. (4) ХІІ... and (е) Х|, when 


"n 


(a) From Problem 11.3. we have ||Х|„ = V(X. X) w = V59. 
(5) ХІ. = VX-X = VE - XO 10) = УЗ. 

(с) XI}, 2n = il + i] = VŽ + I = 2.414, 

(4) XI], = max (|! — il. li) = max (V2.1) = V2. 

(е) ПХ], = 01 Д p * = (5)' * = 1.495. 


12.4 Find the distance between X and Y with respect to (a) the Euclidean norm, (b) the 
inner-product norm with respect to W. and (c) the /, norm when X, Y, and W are as given in 
Problem 12.1. 


For these vectors, X - Y = [-3. -3]'. so 
(a) X = үй. = VC 3 * (73Y + (73. = V7. 
(b) WX- Y —6. - 6. 6|". and 
IX- Yl, = VOCE Y)-W(X - Y) = VC-6XC76) + (-6C-6) (000—6) = 10.392 
(€) IX. Yl, (0 3731-375] 31) "(0 5 4327. 
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12.5 


12.6 


12.7 


12.8 


Normalize the vector X given in Problem 12.1 with respect to (a) the /, norm, (b). the /, 
norm, and (c) the /, norm. 

Using the results of Problem 12.2, we obtain the normalized vectors (a) |1/V 14. 2/V 14, 3/V T]: 
(b) [1/6, 2/6, 3/6]': and (c) [1/3, 2/3. 1]'. Each of these vectors is a unit vector with respect to its 
associated norm. 


Show that the /, norm for vectors does not extend to a matrix norm. 
The /, norm is simply the largest component of a vector in absolute value, and its extension to 
matrices would be the largest element of a matrix in absolute value. That is, 
[А = тах (la, 
КИ 


Consider the matrices 
ES E NE 2] 
a=B=[} i] for which AB E 3 


We have ||A]| = |81 = 1. but |АВ|| = 2. Since condition MS is violated, the proposed norm is not a 
norm. 


Extend the /, vector norm to a matrix norm, and use it to compute the norm of 


_[4+i3 -7 
^-| 3 й 


The /, norm is the sum of the absolute values of all the components of the vector; its extension to 
the matrix А = [a,] is the sum of the absolute values of all the elements of the matrix. That is, 
ПА[, = £7, E7., la,|. This norm automatically satisfies conditions МІ through M4 because they are 
ide І to conditions NI through N4. In addition, for two n X n matrices А and B, the extended norm 
gives us 


inni, + X [5 оа, | 5 X nat 
=$ Ў Ў Š laslonl= (5 Š toal)(S, Ул) = UAB, 


ow 


Thus, condition MS is satisfied and this extension is a matrix norm. Applying it to the given matrix, we 
calculate 


(All = [4 + 13| [77] + [3] + id =5+74+344=19 


Calculate the (a) Frobenius norm, (b) L, norm, (c) L, norm, and (d) spectral norm for the 

matrix A of Problem 12.7. 

(а) АЙ, = (4 + 3]? + |-7]? + [DI Jia) = VI+ 39 € 9 T6 = 9.950. 

(b) [Al], = тах(|4 + i3| + [3], |-7] + ]i4]) = max(5 +3,7 +4) = 11 

(с) All. = max({4  i3| + |—7|, [3] + 124) = тах(5 + 7.3 + 4) = 12. 

(d) We compute 
ay [4-83 3 704483 -77_ 34 —28 + i33 
ТЕР Sala” i lacus es 


which has the characteristic equation A^ — 99A + 337 =0 and the eigenvalues A, = 95.470 and 
А, = 3,530. Thus, [A], = max(V 95.470. V3.530) = 9.771. 
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12.9 Calculate the (a) Frobenius norm, (Б) L, norm, (c) L, norm, and (d) spectral norm for 


(а) АЈ], = (TY + C74) + (0)? + C72) + (79) + (0) + (0)? + (0)? + (79) * = 13.638. 
(b) А, = max({|7| + [-2] + [0], |-4] + |—6] + [0], 0] + 01 + 1-9) = max(9, 10, 9) = 10. 
(с) lAl]. = тах(|7| + |-4| + 101, 1—21 + |-- 6| + |0], |0] + [0] + |—9]) = max(11, 8, 9) = 11. 
(d) Here we have 


Ө 65 10 0 
A'A-A'A-|10 40 0 
0 08 


which has eigenvalues 68.5078, 36.4922, and 81. Thus, ||A||, = max(V 68.5078, V36.4922, V8) = 9. 


12.10 Prove that an induced norm generated by (12.1) is a matrix norm. 


Let A and B denote arbitrary n X n matrices, and X and Y arbitrary n-dimensional vectors. We are 
given a vector norm that satisfies conditions N1 through №, and we wish to show that a proposed matrix 
norm defined by (12.1) satisfies conditions M1 through MS. For clarity we subscript the vector norm 
with V and the matrix norm with M. 


(мї: ||A]||,, is the maximum of nonnegative quantities ||AX||, and must be nonnegative. 

(М2): If A = 0, then AX = 0 for all vectors X, and ||A|[,, = max(|AX||,) = max(0) = 0. If A #0, then A 
must contain at least one nonzero column. Designate the first such column as column i, and 
construct Y by setting its ith component equal to 1 and all other components equal to 0. Now 
[|АҮ||, and [[Y||, are positive, and since Y/||Y||, is a unit vector, 

AY |v 

Mil, 

(M3): |с||,, = max(J|cAXII,) = тах(|с||АХ||,) = |с| max(||AX|},) = ІА. 

(M4): |А + Bl], = max{||(A + B)X||,} = max(||AX + BX||,) 

= max([|AXI|, + ||BX||,) 
= тах(|АХ(|,) + тах(|ВХ||,) = All + ВІЇ, 


(М5): If B=0, then АВ = 0, ||ABI|,, = (18|, = 1011, = 0, and the desired inequality is trivially true. If 
B #0, we can restrict our attention to those unit vectors X for which BX #0, since all others 
have no influence on the norm, Then 


Alle = mx (ARI = АЭ ED 


ТҮЙ, 


v 


WABI, = пах [АВУХ],} = max (ACON) = max {LRD hexi} 
ПАСВХ)||,, BX 
= apes ах e cox = max, (А ra.) es mto 


AY.) a ,CIIBXILO 
vex ex 


= max (АУ max (ЇВХЇ,) = [Allu IBI 
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12.11 


12.12 


Show that the /, vector norm induces the L, matrix norm under Eq. (12.1). 
Set |[Al], = тах (||AXI|,). Then |[Al], is a matrix norm as a result of Problem 12.10. Denote the 
їхї т 


columns of А as vectors A,, Az, . . . , A,, respectively, and set 


H= max, (S fal) = max ЧАЙ) 
We wish to show that [[A||, = H. 
For any unit vector X = [x, x;.... x, ]". 
WAX}, = lx, A, + х,А, +++ tall 
Sle Alh + ПА + + lln Anh = on IAM, + ЛА +, А 
ж [х\н + Ix) Ht не Ales] + le) +++ bx D = АХ, =н 


Thus, ЙА = max CIIAXII,) = max (Н) = H @) 


[ID 


But for unit vectors Y, (k=1,2,...,m) having a 1 as the kth component and Os as all other 
components, 


All, = max (AX) = АҮ, |1, = lAs, 
en 


so А, = max (II, Il) = H (2) 


Together, (1) and (2) imply the desired equality. 


Show that the /„ vector norm induces the L, matrix norm under (12.1). 


Set All. = ymax CIAXIL). Then ||А||„ is a matrix norm as a result of Problem 12.10. 
Now к 


H= max (о) 


We wish to show that [|A|]. = H. This equality obviously holds if A = 0, so we consider only nonzero 
matrices in what follows. For any unit vector X =(x,,x,,-..,x,]’, we have 


mx a,x) = ‚де (5, 15) 
(Š ie,t.) = m (S1) = 


„тах CIAXIIL) = max (Н) = Н a) 


ШЕ 


Thus, Hall. 


' 


Corresponding to the kth row of A (k = 1,2,..., п), define a vector Y, =[y}, y... у] by 


setting 


1 when а,, =0 
e 2d d, 
| =] when a,, #0 
lal 


Then ||Y, ||. = 1, and the kth component of AY, is E7., |a,,|. Also for each k, 
lall. = max (АХ) = АҮ... 


The last term on the right is the maximum component in absolute value of АҮ, , which is at least as large 
as the absolute value of the kth component of AY,. Hence, |А||.. = E7., [а,,| for all k, and 
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12.14 


12.45 


12.16 
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Ml. = max (È la!) =н o 


Together, (1) and (2) imply the desired equality. 


Show that an induced matrix norm with its associated vector norm satisfy the compatibility 
condition ||AY|| = АІ. 
The inequality is immediate when Y = 0. For any nonzero vector Y, Y/||Y|| is a unit vector, and 


Шы] 


IAI max Пах e [ar] = Тү. 


Show that |А|| = max (АХ) = max(||AX|}/||X| 
Set H = тах(АХ/ Х|). We must show that [|All = H. First, we have 
lax] ) = ma( Пахі! ) =H 


А! = max АХ) = max ( 


Ix} oN |X| 7 
where the inequality follows from taking the maximum over a larger set of vectors. Thus, 
ПА s H a) 
It follows from Problem 12.13 that 
à АХ АХ ү _ E 
H= max( Terr) = mas (Сту) mast Al 
so |All = H (2) 


Together, (1) and (2) imply the desired inequality. 


Show that the Frobenius matrix norm is compatible with the Euclidean vector norm. 


For any n X n matrix A and n-dimensional vector X, 


2 
X lith component of AX}? = > | a,x 
d 


“л ilies 


ПАХ[2 = 


Using the subscript i to designate the ith row of a matrix and employing the Schwarz inequality (see 
Chapter 11), we have 


Хал А-х = Kal ЮР 


< (AT AD G3) = (AT A039 «(S la, P)(È һу) = (È аг) 
Therefore, пахи = (X Š lo, P)ixit = WANE itt 


We obtain the required inequality by taking square roots. 


Show that any matrix norm has a vector norm with which it is compatible. 


Let ||All, designate an arbitrary matrix norm on л х п matrices. If Y is an arbitrary but fixed 
n-dimensional, nonzero column vector, then the function f(X) = ||XY"||,, satisfies all the properties of a 
vector norm on the set of all n-dimensional column vectors X. Furthermore, 


КАХ) = |(AX)Y"IL, = ПАСХУ), = [АЙ IXY Un = ПА, SOO) 
so ||Al|,, is compatible with the vector norm f(X). 
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12.17 


12.18 


12.19 


12.20 
12.21 


12.22 


12.23 


12.24 


12.25 
12.26 


12.27 


Determine bounds on the eigenvalues of 
1 


07 8 7 
75 6 5 
A=l 36 10 9 
7 5 9 10 


The row sums and column sums are both 32, 23, 33, and 31, so [|A||, = [|А||„ = 33. The Frobenius 
norm is ||А[|, = 30.5450. It follows from (12.3) that с(А) $33 and о(А) = 30.5450, from which we 
conclude that every eigenvalue must be no greater than 30.5450 in absolute value. (See also Problem 
20.8.) Of course, other norms not considered here might place a still lower bound on the eigenvalues of 
A. 


Prove that с(А) = ||A|| for any matrix norm. 


Let A be an eigenvalue of A for which |A| = o(A), and let X denote a corresponding eigenvector. 
Construct а matrix B having each of its columns equal to X. Then AB = AB, and for any matrix norm 


ТАВ! = |] ABI] = [JABI] = ANB} 
Since B is not a zero matrix, it follows that |A| = ||A||. But |A| = с(А), so o(A)  [JAll. 


Supplementary Problems 


Determine the /, norm of each of the following vectors: 


1 1 2 1 4 
ex-[ ex-| | ez-| | ө) ev-[i] 
0 -3 -4 0 4 


Determine the /, norms of the vectors in Problem 12.19. 
Determine the /„ norms of the vectors in Problem 12.19. 
For each of the vectors in Problem 12.19, determine the inner-product-generated norm with respect to 
0 11 
W-|1-21 
3 02 


For the vectors given in Problem 12.9, determine (а) |Z- Yl; — (b) IZ- Yll; — (c) IIZ- Ylh; 
(a) |U -Vlls; (e) 10-У|,: and (f lU- VII. 


Determine the Euclidean norm of 

i -5 0 1+2 
exe] ev-[2] ez-[] «о [58] 
Determine the /, norms of the vectors in Problem 12.24. 
Determine the J, norms of the vectors in Problem 12.24. 


Determine the /, norms of the vectors in Problem 12.24. 
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12.28 


12.29 


12.30 


12.31 


12.32 


12.33 


12.34 


12.35 


12.36 


12.37 


12.38 


12.39 


12.40 


12.41 
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Determine the Frobenius norms for the following matrices: 


oba з] oR eb) eB 


Determine the L, norms of the matrices in Problem 12.28. 
Determine the L, norms of the matrices in Problem 12.28. 
Determine the spectral norms of the matrices in Problem 12.28. 
Prove that for any induced matrix norm, ||I]| = 1. 

Show that the Frobenius matrix norm satisfies condition MS. 


Prove the Pythagorean theorem for an inner-product-generated vector norm; that is, prove that if 
(X, Y), 70, then [X + У], = XII, + ҮЙ» 


Using the L,, L,, and Frobenius norms, determine an upper bound on the spectral radius for each of the 
matrices in Problem 12.28. 


Determine the spectral radii of matrices (a), (Б), and (c) of Problem 12.28. 


Determine the spectral radius of 


Prove that a(A") = o(A). 


The condition number of a square matrix A with respect to a matrix norm is 


ЈАША ^if A is nonsingular 
ы if A is singular 


c(A) = { 
Determine the condition numbers of the matrices in Problem 12.28 with respect to the Frobenius norm. 
Show that the condition number of an identity matrix is unity for all induced matrix norms. 


Show that the condition number as defined in Problem 12.39 cannot be less than 1. 


Chapter 13 


Hermitian Matrices 


NORMAL MATRICES 


The Hermitian transpose of a matrix A, denoted А", is the complex conjugate transpose of A; 
that is, A" = А", A matrix A is normal if 


AA" = АНА (13.1) 
(See Problem 13.1.) Normal matrices have the following properties: 
Property 13.1: Every normal matrix is similar to a diagonal matrix. 


Property 13.2: Every normal matrix possesses a canonical basis of eigenvectors which can be 
arranged to form an orthonormal set. 


(See Problems 13.7 through 13.9.) 


HERMITIAN MATRICES 


A matrix is Hermitian if it equals its own Hermitian transpose (or complex conjugate transpose); 
that is, A is Hermitian if 


A = A" (13.2) 


The sum of Hermitian matrices is Hermitian, as is the product of a Hermitian matrix with a real 
scalar. A Hermitian matrix is also normal, because AA” = AA = AYA. Therefore, Hermitian matrices 
possess Properties 13.1 and 13.2. In addition, 


Property 13.3: Тһе eigenvalues of a Hermitian matrix are real. 


Property 13.4: If a Hermitian matrix A can be reduced to upper triangular form U using only 
elementary row operations of the third kind (E3), then the diagonal of U contains 
the same number of zeros, the same number of positive values, and the same 
number of negative values as the eigenvalues of A. 


Property 13.5: An nx n matrix A is Hermitian if and only if (AX, X) is real for all (real and 
complex) n-dimensional vectors X. 


(See Problems 13.4, 13.11, and 13.19.) 


REAL SYMMETRIC MATRICES 


A matrix is symmetric if it equals its own transpose. A symmetric matrix that contains only real 
elements is Hermitian and, therefore, normal. Consequently, real symmetric matrices possess 
Properties 13.1 through 13.5 as well as the following: 


Property 13.6: The eigenvectors of a real symmetric matrix can be chosen to be real. 
(See Problem 13.15.) 


THE ADJOINT 
The adjoint of an n X m matrix A is an m X n matrix A* having the property that 
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(X, AY), = (A*X. Y) w (13.3) 


for all m-dimensional vectors Y and n-dimensional vectors X, where the inner product is as defined 
in Chapter 11. The adjoint always exists and it is 


At = (W^W) 'A"(w'w) (13.4) 
For the special case W =I (the Euclidean inner product), (13.4) reduces to 
A* = A" (13.5) 


(Sec Problems 13.16 through 13.18.) Adjoints satisfy the following identities: 
(А1): (A*)* =A. 

(А2): (A + В)‘ - A* + В". 

(АЗ): (AB)* = В*А*. 

(А4): (сА)* = cA* for any scalar c. 


SELF-ADJOINT MATRICES 


A matrix A is self-adjoint if it equals its own adjoint. Such a matrix is necessarily square, and it 
satisfies the identity 


(X. AY) y = (AX, Y)w (13.6) 


for all vectors X and Y of appropriate dimension. A matrix is self-adjoint with respect to the 
Euclidean inner product if and only if it is Hermitian. 


Solved Problems 


13.1 Determine which of the following matrices are normal: 


12 3 26 -3 
[1] [4-4] etn [ii 
3-5 0 * -63 2 


A is not square, so it cannot be normal. B is real and symmetric and, therefore, normal. € is 
Hermitian and, therefore. normal. D is normal because 
=3 
6|-D"p 
2 


26 -3 2 3 =6 49 0 0 23 -6 2 
nDD'- 32 6j 62 3f=| 0 4 Of=| 62 3H 3 
38 3 bx 2 0 0 49 -3 6 211-6 


13.2 Show that A"A and AA" are normal for any matrix A. 


шю о 


(А А)” =) (A'A)! = (А) (А) = A'A = А"А 
and (AA")" = (AA!) = (ААГ) = (A')'(A)' = AA’ = AA" 


Both A"A and AA“ equal their own Hermitian transposes, so they are Hermitian and, therefore, normal. 
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13.3 


13.4 


13.5 


13.6 


13.7 


Prove that the eigenvalues of AYA are nonnegative. 


If A is an eigenvalue of A"A, then there must exist a nonzero eigenvector X associated with A 
satisfying the equality A"AX = AX. For the Euclidean inner product, it follows from Property 11.1 and 
Eqs. (13.3) and (13.5) that 


0< (AX, AX) = (A*AX, X) = (A"AX, X) = (AX, X) = A(X, X) @) 


Since X is an eigenvector, it is nonzero and we may infer from Property 11.2 that (X, X) is positive. 
Dividing (1) by (X, X) yields А20. 


Prove that the eigenvalues of a Hermitian matrix are real. 


Let A denote an eigenvalue of a Hermitian matrix A, and let X denote a corresponding eigenvector. 
Then, under the Euclidean inner product, 


A(X, X) = (AX, X) = (AX, X) = (X, A*X) = (X, AX) = (X, AX) = (X. AX) =А(Х,Х) (1) 


Since X is an eigenvector, it is nonzero and so too is (X, X). Dividing (1) by (X, X) gives us A= A, 
which implies that A is real. 


Show that if X is an eigenvector of a normal matrix A corresponding to eigenvalue A, then X is 
an eigenvector of A" corresponding to А. 


Using the Euclidean inner product and (13.1), we obtain 
(AX, AX) = (A*AX, X) = (A“AX, X) = (AA"X, X) = (АХ, A*X) = (A"X, A"X) 


It then follows that 


= (0,0) = (AX – AX, AX - AX) 

= (AX, AX) - À (AX, X) - A(X, AX) + (AX, AX) 

= (АНХ, АНХ) — A(X, A*X) - A(A*X, X) + AÀ (X, X) 
= (A"X, АНХ) — A(X, A"X) – A(A"X, X) + (AX, AX) 
= (АЧХ - AX, A"X - АХ) 


Thus, АХ — AX = 0, which implies that X is an eigenvector of A" corresponding to А. 


Show that eigenvectors corresponding to distinct eigenvalues of a normal matrix are orthogon- 
al with respect to the Euclidean inner product. 


Let A, and A, be two distinct eigenvalues of a normal matrix A with corresponding eigenvectors X, 
and X,. Then AX, = A,X, and AX, = A,X,, and А": X,= ay X, as a result of Problem 13.5. Furthermore, 
(А-А) (Х.Х) 74,06, X3) А, (Х,,Х,) = (AK, X2) = (X АХ.) 
= (АХ,,Х,) - (X,,A"X,) = (AX, X3) - (X,, A*X,) 
= (AX, X,) - (AX,,X,)=0 


Since A, # А, it follows that (X;, X;) = 0. 


Show that a set of linearly independent eigenvectors of a normal matrix can be arranged to 
form an orthonormal set of eigenvectors. 


Eigenvectors corresponding to distinct eigenvalues of a normal matrix are orthogonal by Problem 
13.6, and they remain orthogonal eigenvectors if each is normalized. Therefore, we need only show that 
linearly independent eigenvectors corresponding to the same eigenvalue can be so arranged. But this is 
easily accomplished by the Gram-Schmidt orthogonalization process. Because this process forms linear 
combinations from a given set (in such a way as to produce orthonormal vectors), it follows from 
Problem 7.12 that the resulting vectors will remain eigenvectors. 
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13.9 
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Determine a canonical basis of orthonormal eigenvectors with respect to the Euclidean inner 


product for 
2 жь 
A=| 2 2 -2 


2 -2 6 


The matrix is real and symmetric and, therefore, normal. The eigenvalues for A are 0, 2, and 8, and 
a corresponding set of eigenvectors is 


[J «d «d 


Since each eigenvector corresponds to a different eigenvalue, the vectors are guaranteed to be 
orthogonal with respect to the Euclidean inner product. Dividing each vector by its Euclidean norm, we 
Obtain the orthonormal set of eigenvectors 


~1/V2 1УЗ -1/М6 
Q,*| 1/7? Q: =| 1/V3 Q, =| -1/V6 
0 1/УЗ, 2% 


Determine a canonical basis of orthonormal vectors with respect to the Euclidean inner 
product for 


з -i2 0 R 
i2 d o0 
0 = 3 =2 
-2 0 -2 Ш 


A= 


A is Hermitian and, therefore, normal. Its eigenvalues are 5, 5, —1, and —1, with corresponding 
eigenvectors 


i i2 i -i 
NE -1 B -1 
xc oy of =] X=! 9 
0. 1 0. 1 


Since X, and X, correspond to one eigenvalue, and X, and X, to another, each of the first two vectors is 
guaranteed to be orthogonal to the latter two. Applying the Gram-Schmidt orthogonalization process to 
the first two vectors, we obtain 


ilv3 УЗ 
-1МЗ 0 

Ф| ia| | уз 

0 1/73. 

Applying the Gram-Schmidt orthogonalization process to the latter two vectors, we calculate 

ié -ilv6 
_| 2/v6 i 0 

Q% =] iva 9.-| iva 

0 26 


The set (Q,,Q,,Q;.Q,} is a canonical basis of orthonormal eigenvectors for A. 
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13.10 Verify Property 13.4 for the matrix in Problem 13.8. 


The eigenvalues for the matrix are 0, 2, and 8, so it has one zero eigenvalue and two positive 
eigenvalues. Reducing the matrix to upper triangular form using only elementary row operations of the 


third kind, we obtain 
2 2 22 
d x» 2 
-2 -2 6. 


2 2 -2 Adding —1 times the 
=| 0 9 0 first row to the 
= second row 


00 0 to the third row 


f 2 3] Adding the first row 
9100 4 


This last matrix is in upper triangular form, and the diagonal elements consist of one zero and two 
positive numbers. 


13.11 Verify Property 13.4 for the matrix in Problem 13.9. 


The eigenvalues for that matrix are 5, 5, — 1, and —1, which consist of two positive and two negative 
numbers. Reduced to upper triangular form via elementary row operations of the third kind, the matrix 
becomes 


3 -i2 0 i2 
0 -1/3 -2 4/3 


0 0 15 -10 
0 0 0 -5/3 


The diagonal elements of this matrix also consist of two positive and two negative numbers. 


13.12 Show that if A is Hermitian, then A — cl is also Hermitian for any real scalar c. 


(&-d)' = (A~ cl)’ = (A- cl)" =A" - (с) = A- cl 


13.13 Prove that a Hermitian matrix is similar to a diagonal matrix. 


We need only show that a Hermitian matrix A does not possess any generalized eigenvectors of rank 
2. This, in turn, implies that it possesses no generalized eigenvectors of rank greater than 2, because 
otherwise we could form a chain and obtain, as part of the chain, a generalized eigenvector of rank 2. 
Thus, all generalized eigenvectors have rank 1, and it follows from Chapter 10 that the Jordan canonical 
form of A is a diagonal matrix. 

Assume that X is a generalized eigenvector of rank 2 corresponding to the eigenvalue A. Then 


(A- AD/X-0 
and (А ADX #0 @) 
We may infer from Problem 13.4 that A is real and from Problem 13.12 that A — AI is also Hermitian, 


Thus, 
0= (X,0) = (X, (A – AD'X) = ((А – AD*X, (A - ADX) 
= ((А — AD^X, (A — АХ) = ((A — АЮХ, (A — ADX) 2) 
We conclude from (2) and Property 11.2 that (A — AI)X =0. But this contradicts (1), so X cannot be a 


generalized eigenvector of rank 2. (See Problem 15.11 for the generalization of this result to all normal 
matrices.) 
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13.14 Show if a matrix is upper triangular and normal, then it must be a diagonal matrix. 


Let A = [а,| be an n X n upper triangular matrix that is also normal. Then a, = 0 for i j. We show 
sequentially, for = 1, 2,...,п — 1, that a, 7 0 when i<j. Since 


АНА = AA" (1) 


it follows from equating the (1,1) elements of the two products in (1) that 


ала = anā, +B asd, 
£ 


so that 0= Xla,l 
2 
Thus, 
a,=0 (j=2,3,...,n) (2) 


Next, equating the (2,2) elements of the two products in (1) and using (2), we obtain 
js 
so that 0- X lal 
i 


for which we infer that 
ay70 (ў=3,4,...,п) 


Continuing in this manner—working with each successive diagonal element in turn—we find that all 
elements above the diagonal of А must be zero. Thus, all nondiagonal elements of A are zero, and A is а 
diagonal matrix. 


13.15 Show that the eigenvectors of a real symmetric matrix can always be chosen to be real. 


Let X be an eigenvector of a real symmetric matrix A corresponding to the eigenvalue A (which 
must be real as a result of Problem 13.4). If the components of X are all pure imaginary, then Y = iX is 
real. and 


AY = A(iX) = (АХ) = (АХ) = A(iX) = AY 


so Y is also an eigenvector of A corresponding to A. If the components of X are not all pure imaginary, 
then Y = X + X is not zero but is real, and 


AY = A(X + X) AX + AK = AX + AX = AX + AX 2 AX + AX =A(X+ X) AY 


and Y is a real eigenvector of A corresponding to A. 


13.16 Determine the adjoints of the following matrices with respect to the Euclidean inner product: 


3 aao 
4-i5 6 


SEHE 


rz 
a-[3 d B- (15, -5, 2 i3] c=[ 


In each case, the adjoint is the Hermitian transpose of the given matrix, as provided by (13.5); 
hence, 
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13.17 


13.18 


13.19 


C and E are self-adjoint because both are Hermitian. 


Determine the adjoint of A under an inner product with respect to W, where 
Fe. ] -[ i4 A 
^=[ apa] ma [5 | 
Using (13.4), we calculate 
нуу [14 -1][i4 S] [17 -i21 

wee? ДИ RUD жш ] 

„ениш | Ж 01][2. 3 17 -i2 

and Pe (WW) AWW) m Le n l[o еа 26 | 


_ [ -2,077 + 11,764 2,184 + 12,574 
7| 1,428 + 11,680 2,082- 11.768 


Derive (13.4). 
For an arbitrary inner product defined with respect to a nonsingular matrix №, we have 
(X, AY) w = (WX): (WAY) = (WX)'(WAY) = X’W’ WAY 
and (A*X, Y), = (WA*X)* (WY) = (WA*X) (WY) = X'(A*) w WY 
The two inner products are equal by (13.3), which implies that 
X7W’ WAY = X'(A*) w WY 
or X'(W'WA - (А?) ММ) =0 
This last equation is valid for all vectors X and Y if and only if W" WA — (A*)’W’W = 0, from which we 
infer that (W'W)A(W^W) '' =(A*)’, and 
А” = (WWW W) T = (WW) ')'A'(W"W)' = ((W"W)") "A'(W"w) 
= (W"W)"A'(W"W) = (W”W) 'A"(W"w) 


Prove that if an n x n matrix A is self-adjoint, then (AX, X)y is real for all n-dimensional 
vectors X. 
Using Property 11.3, we have 


(X, AX) w = (AX. X) w a) 


But if A is self-adjoint, then also 
(X, AX) w = (AX, X)w (2) 


It follows from (1) and (2) that (AX, X) w= (АХ, X)w, which implies that the inner product is real. 
For the special case of the Euclidean inner product, this result reduces to Property 13.5 for Hermitian 
matrices. 
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13.20 


13.21 


13.22 


13.23 


13.24 


13.25 


13.27 


13.28 


13.29 


13.30 
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Supplementary Problems 


Determine which of the following matrices are Hermitian: 


Determine which of the matrices in Problem 13.20 are normal. 
Find a canonical basis of orthonormal vectors for matrix F in Problem 13.20. 


Find a canonical basis of orthonormal vectors for 


сюесе 
coooc 


Verify Property 13.4 for (a) matrix F of Problem 13.20; (b) matrix J of Problem 13.23; and (c) matrix K 
of Problem 13.24. 


Determine the adjoint of matrix D of Problem 13.20 for (a) the Euclidean inner product and (5) the 
inner product with respect to 
11 
w [0 1) 


Determine the adjoint of matrix Е of Problem 13.20 for (a) the Euclidean inner product and (b) the 
inner product with respect to 
100 
#=|0 2 1 
002 


Determine the adjoints with respect to the Euclidean inner product for matrices A, B, and G of Problem 
13.20. 


Prove that the sum of Hermitian matrices is Hermitian. 


Prove that if A and B are Hermitian and AB = BA, then AB is Hermitian. What does this imply about 
powers of Hermitian matrices? 
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13.31 


13,32 


13.33 


13.34 


13.35 


13.36 


13.37 


13.38 


Prove that the diagonal elements of a Hermitian matrix must be real. 
A matrix A is skew-Hermitian if А = —A". Show that such a matrix is normal. 
Show that if A is skew-Hermitian, then iA is Hermitian. 


Show that if A is an n Xn skew-Hermitian matrix, then (AX, X) is pure imaginary for every 
n-dimensional vector X. 


Show that if A is skew-Hermitian, then every eigenvalue of A is pure imaginary. 
A matrix А is skew-symmetric if А = — A". Show that a real skew-symmetric matrix is skew-Hermitian. 


Show that any real matrix can be written as the sum of a symmetric matrix and a skew-symmetric matrix, 
and show that any complex-valued matrix can be written as the sum of a Hermitian matrix and a 
skew-Hermitian matrix. 


Prove that any well-defined function of a Hermitian matrix is Hermitian. 


Chapter 14 


Positive Definite Matrices 


DEFINITE MATRICES 
An n x n Hermitian matrix A is positive definite if 


(AX, X) >0 (14.1) 
for all nonzero n-dimensional vectors X; and A is positive semidefinite if 
(AX, X) z0 (14.2) 


If the inequalities in (14.1) and (14.2) are reversed, then A is negative definite and negative 
semidefinite, respectively. 

The sum of two definite matrices of the same type is again a definite matrix of that type, as is the 
Hermitian transpose of such a matrix. Positive (or negative) definite matrices are invertible, and their 
inverses are also positive (or negative) definite. 


TESTS FOR POSITIVE DEFINITENESS 
Each of the following three tests stipulates necessary and sufficient conditions for an n X n 


Hermitian matrix A to be positive definite. That is, a Hermitian matrix A is positive definite if it 
passes any one of these tests. 


Test 14.1: A is positive definite if and only if it can be reduced to upper triangular form using only 
elementary row operations E3 and the diagonal elements of the resulting matrix (the 
pivots) are all positive. 


Test 14.2: A principal minor of A is the determinant of any submatrix obtained from A by deleting 
its last А rows and k columns (А = 0, 1,... ‚п — 1). A is positive definite if and only if all 
its principal minors are positive. 


Test 14.3: A is positive definite if and only if all its eigenvalues are positive. 


The following tests stipulate necessary conditions for an n x n matrix A = [а,] to be positive 
definite. A Hermitian matrix that fails any one of these tests is not positive definite, but no 
conclusions can be drawn about a Hermitian matrix that passes them. 


Test 14.4: The diagonal elements of A must be positive. 
Test 14.5: The element of A having the greatest absolute value must be on the diagonal of A. 
Test 14.6: 2,27 |a," (1 j). 


(See Problems 14.1 through 14.11.) All these tests may be changed to tests for positive semidefinite- 
ness by replacing the word positive with nonnegative and replacing > with =. They can also be used 
to test a matrix A for negative definiteness or semidefiniteness if they are applied directly to ~A. This 
is equivalent to replacing the word positive (or nonnegative) with negative (or nonpositive) in tests 
14.1 through 14.4; Tests 14.5 and 14.6 are applied as stated. 


SQUARE ROOTS OF MATRICES 


The square root of a matrix A is a matrix A"? having the property that А = A'"A'". If A and A’? 
are both required to be positive definite or positive semidefinite, then A"? is unique, and the square 
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root is a well-defined function. In such cases it may be calculated by the methods given in Chapters 8 
and 10. (See Problems 14.13 and 14.14.) 


CHOLESKY DECOMPOSITION 
Any positive definite matrix A may be factored into 
A-LL" (14.3) 


where L is a lower triangular matrix having positive values on its diagonal. Equation (14.3) defines 
the Cholesky decomposition for A, which is unique. 

The following algorithm generates the Cholesky decomposition for an л X n matrix А  [a,] by 
sequentially identifying the columns of L on and below the main diagonal. It is a simplification of the 
LU decomposition given in Chapter 3. 

STEP 14.1: Initialization: Set all elements of L above the main diagonal equal to zero, and let 
lı = Уа. The remainder of the first column of L is the first column of A divided by 
lı» Set a counter j = 2. 

STEP 14.2: If j7 n + 1, stop: the algorithm is complete. Otherwise, define L; (i=j, j * 1,..., n) 
to be a column vector of dimension j — 1 whose components are, respectively, the first 
j - 1 elements in the ‘th row of L. These elements have already been computed. 


STEP 14.3: Compute 


= Va, (1,1). 


STEP 14.4: If j— п, skip to Step 14.5; otherwise compute the jth column of L below the main 
diagonal: For each i 7 j +1, j+2,...,n, compute 


Саая) 
ы 1, 
STEP 14.5: Increase j by 1, and return to Step 14.2. (See Problems 14.15 and 14.16.) 


Solved Problems 
14.1 Use Tests 14.1 through 14.3 to verify the positive definiteness of 


6 Z -2 
A=| 2 6 -2 


2 -2 10 


6 2 d Adding —1/3 times the first row to 
Test 14.1: —| 0 16/3 -4!/3 the second row 
z2 =2 10 
6 2 -2 Adding 1/3 times the first row to 
0 16/3 -4/3 the third row 
10 -4/3 28/3 
6 2 E Adding 1/4 times the second row 
0 16/3 -4/3 to the third row 
э 0 273 


Since the pivots, 6, 16/3, and 27/3, are all positive, the matrix is positive definite. 
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Test 14.2: The principal minors of A are 


T 9 23 
de[s]-6 |$ 2|=36-4=32 апа 2 6 -2 |=288 
ża p 


Since all three principal minors are positive, the matrix is positive definite. 


Test 14.3: The eigenvalues of A are 4, 6, and 12. Since all three are positive, the matrix is positive 
definite. 


14.2 Use all the tests to determine whether the following matrix is positive definite: 
2; 10 -2 
A=| 10 5 8 
-2- Ж П 


2 1490 =2 Adding —5 times the first row to 
>| 0 -45 18 the second row 
=2 8 n 


Test 14.1: 


Since the second pivot, —45, is negative, A is neither positive definite nor positive 
semidefinite. We can also rule out A being either negative definite or negative semidefinite, 
because the first pivot, 2, is positive. 


Test 14.2: det [2] = 2, but 
2 10 
hio 21-9 
50 A is not positive definite. 


Test 14.3: The eigenvalues of A are —9, 9, and 18. Since they are not all positive, the matrix is not 
positive definite. 


Test 14.4: The diagonal elements, 2, 5, and 11, are all positive, so no conclusion can be drawn from 
this test. 


Test 14.5: The element of greatest absolute value is 11, which does appear on the main diagonal. No 
conclusion can be drawn from this test. 


Test 14.6: With i= 1 and j= 2, we have 
2,45, = X5) = 10 < 100 = (10) = |]? 


so A is not positive definite. 


14.3 Determine whether the following matrix is positive definite: 


Ho-3 5 -8 
-3 nn -5 -8 
A=| $ -s 19 0 
-8 -8 0 16 


To use Test 14.1, we reduce A to the following form, using only elementary row operations E3: 


ip .=3 3 =g 
0 112/11 ~40/11 -112/11 
0 0 108/7 0 
0 0 0 0 


Since the pivots, 11, 112/11, 108/7, and 0, аге not all positive, the matrix is not positive definite. 
However, these pivots are nonnegative, so A is positive semidefinite. 
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14.4 


14.5 


14.6 


14.7 


14.8 


14.9 


Determine whether the following matrix is positive definite: 
2 cT 7 
A-|-17 -4 1 
7 1-14 


A is not positive definite because it fails tests 14.4, 14.5, and 14.6: Its diagonal elements are not all 
positive; the largest element in absolute value, —17, is not on the main diagonal; and a,,a,, = —8 is not 
greater than |a,,|* = 289. 


Prove that the diagonal elements of a positive definite matrix must be positive. 


If A has order n X n, define X to be an n-dimensional vector having one of its components, say the 
kth, equal to unity and all other components equal to zero. For this vector, (14.1) becomes 


0< (AX, X) =(AX)-X=a,, 


Prove that if PASI ] is an nxn positive definite matrix, then for any distinct i and j 
(i, f=1,2,..., 0), аца > |а, 


Define X to be an n-dimensional vector having all components equal to zero except for the ith and 
jth components. Denote these as x, and x,, respectively: For this vector, (14.1) becomes 


0« (AX, X) = (AX): X 
Setting x, = —a,,/a,, and x, = 1, we find that the first two terms on the right cancel, and we are left with 


F. 
u 


Q,*taxX taxX RA 


7484 1 
0<—~ +a, = — (-a,a, + a,a,) 
n а, 


The desired inequality follows, since a, is positive (see Problem 14.5) and, because A is Hermitian, 
„=й, 


Show that the largest element in absolute value of a positive definite matrix must lie on the 
main diagonal. 


Assume that the largest element in absolute value does not lie on the main diagonal but rather in 
another location, say the (i, j) position, with i # j. Then |а,| >а, and |a,|>a,,. It follows that 
(а, = laylla;l> a, 


adi; 


which contradicts the result of Problem 14.6. Thus, the assumption is incorrect. 


Prove that the eigenvalues of a positive definite matrix are positive. 
Let A be positive definite with eigenvalue A and corresponding eigenvector X. Then for this X, Eq. 
(14.1) becomes 
0« (AX, X) = (АХ, X) = A(X. X) а) 


Since X is an eigenvector, it is not zero and (X, X) is positive (Property 11.1). Dividing (1) by (X, X), 
we obtain А> 0. 


Prove that if all the eigenvalues of a Hermitian matrix are positive, then the matrix is positive 
definite. 

An n X n Hermitian matrix has a canonical basis of orthonormal eigenvectors (Property 13.2). 
Denote these basis vectors as X,,X,,...,X,, with corresponding eigenvalues А,, 4,,..., À,. Then 


AX, = AX, (91,2, .... n). 
If X is any nonzero n-dimensional vector, then the set (X,, X,,..., X,, X) is linearly dependent 
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(Property 6.1). But the orthonormal eigenvectors are linearly independent (Problem 11.27), so it follows 
from Property 6.2 that there exist constants d,,d,,...,d,, such that 
X-dX,*dX,*-- dX, 
Then AX =d,AX, + d,AX, +--+ d, AX, = а, АХ, +d A,X, +++ 4„А„Х„ 
and (АХ, X) = (d,A,X, + d,A,X, +--+ td AX, dX, dX, + td,X,) 
= |d FA, + 14А MuR, 


because the eigenvectors are orthonormal. Since the eigenvalues are given to be positive, this last 
quantity is positive for any nonzero vector X; thus the matrix A satisfies (14.1) and is positive definite. 


Show that the determinant of a positive definite matrix is positive. 


The determinant of a matrix is the product of its eigenvalues (Property 7.8), and each eigenvalue of 
a positive definite matrix is positive (Problem 14.8). 


Show that all principal minors of a positive definite matrix must be positive. 


Let A be an n х n positive definite matrix, and let B be a submatrix of A obtained by deleting from 
А its last k rows and k columns (k =0,1,..., n — 1). Then B has order (n — k) х (n — k). Let Y denote 
an arbitrary nonzero (n — k)-dimensional vector, and define X to be an n-dimensional vector having its 
first n — k components identical to those of Y and its last k components equal to zero. It follows from 


Eq. (14.1) that 
0< (AX, X) = (BY, Y) 


Since this is true for any nonzero vector Y, it follows that B is positive definite and, from Problem 14.10, 
that det B is positive. 


Show that a positive definite matrix is invertible. 


The determinant of a positive definite matrix is positive and so nonzero (Problem 14.10), and 
therefore that matrix must have an inverse as given by (5.3). 


Find the square root of matrix A in Problem 14.1. 


The eigenvalues of А are 4, 6, and 12, with corresponding eigenvectors [1, —1, 0)”, [1, 1, 1]7, and 
[1, 1, -2]” Thus, by (10.2), A = МУМ`', where 


тї oid 40 0 
M-|-11 1 and  J-j|0 6 0 


01-2 0012 
11 1772 о о үз -36 о 

Also, A"-MJUM^C-|-1 1 10 v6 0 |26 2/6 2/6 
01 -2]0 0 vi2]te 1/6 -2/6 


2.3938 0.39385  —0.33820 
0.39385 2.3938 -0.33820 
—0.33820 -0.33820 3.1259 
Show by example that if the square root of a matrix is not required to be positive definite or 
positive semidefinite, then it is not unique. 


For I р A 


each of the matrices 
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Qno NES! °] -[ 4 
в-[ a] -[70 a] 9-10 1 
has the property that its square is A. Only D is positive definite. 


14.15 Determine the Cholesky decomposition for 


4 i2 -i 
A-|-i2 10 1 
i 1:9 


Since A is a 3 x 3 matrix, so too is L in (14.3). 


STEP 14.1: Set l, = Vá - 2; then l, = —i2/2 = —i and l,, = i/2. Set j = 2. To this point, then, we 
have 


STEP 14.2: Define L; = [- i] and L4 = [i/2]. 
STEP 14.3: Compute 


l3 = Vas - (14.10) = V(0071 =3 


STEP 14.4: Compute 


To this point, we have 
2 0 0 
L=|-i 3 0 
i2 1/2 - 


STEP 14.2: us[2] 
STEP 14.3: Compute b, = Vas (ЕП) = V97172- VES 


Therefore, the complete decomposition is 


4 i -i 2 0 0 2 i -i2 
=й 10 1 |=|-1 3 0 03 1/2 
i 1 9 #/2 1/2 V8&5]LO 0 V8.5. 
14.16 Determine the Cholesky decomposition for 
16 -3 5 -8 


STEP 14.5: Setj=3. 


_|-3 16 -5 -8 
Ада] 5 -5 24 0 
-8 -8 0 21 


Since A is a 4 x 4 matrix, so too is L in (14.3). 


STEP 14.1: Set l, = УЇ6 = 4. Then І, = -3/4 = —0.75; 1, = 5/4 = 1.25; L, = -8/4 = —2. Set j = 2. 
To this point we have 
4 0 
-0.75 - 
L25 cc 
29 = a 


оо 


L= 


гооо 
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STEP 14.2: Define L; = [-0.75]; L; = [1.25]; and L; = [-2]. 
STEP 14.3: Compute 

1, = Vaz, - (L;, L;) = VI6 0.5625 = 3.92906 
STEP 14.4: Compute 


tas RIDERE, A TLO la = оу = -2.41788 
To this point, we have 
4 0 00 
21-075 390906 0 0 
125 -103396 - 0 
-2 -24178 - - 
STEP 14.5: Increase j by 1 to j=3. 
7 TE 1.25 " =2 ] 
PSP. 1623 це[ ia] tm [ aar 
STEP 14.3: Is = Vas = (4. = У 12631573 = 4.62260 


STEP 14.4: Compute 


224 7 (La Ly 


„= "m 
To this point, we have 
4 0 0 0 
1219.75 39096 0 о 
| 1.25 1.03396 4.62260 0 
-2 2.41788 0 - 
STEP 14.5: Increase j by 1 to j = 4. 
STEP 14.2: ы - [-2, -2.41788, 0]” 
STEP 14.3: lag = Và - (ы. La) = V21 — 9.84614 = 3.33974 
Finally, we have А = LL”, where 
4 0 0 0 
L-.|-075 39206 0 0 


1.25 —1.03396 4.62260 0 
-2  -241788 0 3.33974. 
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Supplementary Problems 


14.17 Determine which of the following matrices are positive definite and which are positive semidefinite: 


3. L —-4 ОЕ Ж: Srt 
A-| 1 3 -1 B-| 1 1-1 c=|0 52 
poc 55 ж Жы З, 005 


25 1+12 3-i2 
і F=|1+i2 7 2*i 


3-i2 2+1 9 


9-3 0-3 01222 
-vo* 364 -1 34 2 
Gr ё nei] F 2 403 d 
-3 0 -3 6. -1 21 1 


14.18 Find the square root of matrix A in Problem 14.17, given that its eigenvalues are 2, 3, and 6. 
14.19 Find the square root of matrix B in Problem 14.17, given that its eigenvalues are 0, 1, and 4. 
14.20 Find the square root of 


25 124 
-i 25 


K= 
14.21 Find the Cholesky decomposition for matrix А in Problem 14.17. 
14.22 Find the Cholesky decomposition for matrix E in Problem 14.17. 
14.23 Find the Cholesky decomposition for matrix G in Problem 14.17. 
14.24 Prove that the sum of two positive definite matrices is positive definite. 
14.25 Prove that if A is positive definite, then so too is А“. 
14.26 Prove that if A is positive definite, then so too is AW’. 
14.27 Prove that if A is positive definite and C is nonsingular, then B = CAC is also positive definite. 
14.28 Show that if A is Hermitian, then e^ is positive definite. 


14.29 Show that the requirement (AX, X) >0 for all complex-valued vectors X of suitable dimension is 
sufficient to guarantee that A be Hermitian (as well as positive definite). 


14.30 Show that there exist nonsymmetric real matrices that satisfy (14.1) for all real-valued vectors of suitable 
dimension. 


Chapter 15 


Unitary Transformations 


UNITARY MATRICES 


A matrix is unitary if its inverse equals its Hermitian transpose; that is, U is unitary if 
u'=U"%=0" (15.1) 


Unitary matrices are normal because UU" = UU! = I- U^'U = U"U. In addition, they have the 
following properties: 


Property 15.1: А matrix is unitary if and only if its columns (or rows) form an orthonormal set of 
vectors. 


Property 15.2: Тһе product of unitary matrices of the same order is a unitary matrix. 


Property 15.3: If U is unitary, then (ОХ, UY) = (X, Y) for all vectors X and Y of appropriate 
dimension. 


Property 15.4: All eigenvalues of a unitary matrix have absolute value equal to 1. 
Property 15.5: The determinant of a unitary matrix has absolute value equal to 1. 


(See Problems 15.2, 15.5 to 15.7, and 15.24.) Unitary matrices are invaluable for constructing 
similarity transformations (see Chapter 10), because their inverses are so easy to obtain. 
An orthogonal matrix is a unitary matrix whose elements are all real. If P is orthogonal, then 


p'=p" (15.2) 


SCHUR DECOMPOSITION 


Every square matrix is similar to a matrix in upper triangular form, and a unitary matrix may be 
chosen to produce the transformation. That is, for any square matrix A, there exists a unitary matrix 
U such that 


U"AU-U 'AU- T (15.3) 


where T is a matrix in upper triangular form. Equation (15.3) is called a Schur decomposition for A. 
Such a decomposition is not unique, even though the diagonal elements of T must be the eigenvalues 
of A. 

The following algorithm for producing a Schur decomposition for an n X n matrix A is iterative; 
it sequentially generates, at each stage, matrices О, and Т, (k = 1,2,..., n — 1). Each matrix О, is 
unitary, and each T, has only zeros below its main diagonal in its first k columns. T, , is in upper 
triangular form, and U=U,U,---U,_, is the unitary matrix that transforms A into T, ,. For 
notational convenience we set T, = A. The kth iteration of the algorithm is: 


STEP 15.1: Denote as A, the (n — k + 1) x (n — k + 1) submatrix in the lower right portion of 
Tio 
STEP 15.2: Determine an eigenvalue and a corresponding unit eigenvector for A,. 


STEP 15.3: Construct a unitary matrix N, which has as its first column the eigenvector found in 
Step 15.2. 
STEP 15.4: For k=1, set U, = Ny; for k>1, set 
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where I,_, is the (k — 1) x (k — 1) identity matrix. 
STEP 15.5: Calculate T, = ОНТ, ,U,. 


(See Problems 15.8 and 15.9.) 
1f A is normal, then the Schur decomposition implies: 


Theorem 15.1: Every normal matrix is similar to a diagonal matrix, and the similarity transforma- 
tion can be effected with a unitary matrix. 
(See Problem 15.11.) 


ELEMENTARY REFLECTORS 


An elementary reflector (or Householder transformation) associated with a real n-dimensional 
column vector V is the л X п matrix 
vv? 
R-1-2.—5; (15.4) 
Пу: 


An elementary reflector is both real symmetric and orthogonal, and its square is the identity matrix. 
(See Problems 15.13, 15.14, and 15.22.) 


SUMMARY OF SIMILARITY TRANSFORMATIONS 


As indicated by (10.1), a similarity transformation requires the computation of an inverse; and 
inversion is a tedious process for all but unitary matrices, whose inverses are their Hermitian 
transposes. If the matrix S in (10.1) is unitary, then A and B are said to be unitarily similar. 

Similarity transformations are important because they preserve many basic attributes of a square 
matrix—in particular, eigenvalues (Problem 10.3)—while yielding matrices that are simpler in form. 
The simplest form is that of a diagonal matrix, and any matrix possessing a canonical basis of 
eigenvectors (Chapter 9) is similar to a diagonal matrix. Normal matrices (Chapter 13) have this 
feature, and they include Hermitian and unitary matrices. The most that can be said of an arbitrary 
square matrix is it is similar to a block diagonal matrix in Jordan canonical form. 

If the matrix S in (10.1) is restricted to be unitary, then the simplest general form that results is 
no longer Jordan canonical form but upper triangular (via Schur decomposition). Normal matrices 
are special in that their Schur decompositions are diagonal matrices. Thus, normal matrices are 
unitarily similar to diagonal matrices. 

In practice, to perform a similarity transformation requires knowledge of eigenvalues and 
eigenvectors, and that information is generally difficult to obtain. Numerical techniques for approx- 
imating these quantities are given in Chapters 19 and 20. 
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Solved Problems 


Determine which of the following matrices are unitary: 


tos Чоё 

А ee 67 2/7 3/7 A 9 

ne V2 t 02 02-02] »-[27 -6/7 -27] cepa 291 cd 
$ 277 3m -6/7 o 


All three are unitary, because the product of each with its Hermitian transpose yields an identity 
matrix. Since the elements of B are real, that matrix is also orthogonal. 


Prove that a matrix is unitary if and only if its rows (or columns) form an orthonormal set of 
vectors. 


Designate the rows of U as U,,U,,...,U,. Then the (i, j) element (= 1,2,...,и; j= 
1,2,..., n) of UU" is 


(UU"), , - U,-D, =(U,,U,) 


If U is unitary, then UU" =1, and this (i, j) element (U,, U,) must be 1 when i = j and 0 otherwise. 
This, in turn, implies that the set (U,, U}, .. . , U,) is an orthonormal set of vectors. (The columns of U 
may be shown to be orthonormal by considering the product U"U instead.) 

Conversely, if the rows (or columns) of a matrix form an orthonormal set, then the argument given 
above may be reversed to show that U is unitary. 


Show that if A is an eigenvalue of an n X n matrix A, then there exists an n X n unitary matrix 
U having as its first column an eigenvector of A corresponding to A. 


If A is an eigenvalue of A, then there must exist an eigenvector X corresponding to A. Setting 
Y = X/||X]| gives us a unit eigenvector of A corresponding to A. 

Consider the set of vectors {Y,E,,E,,...,E,}, where E, (k=1,2,...,m) has a 1 as its kth 
component and all other components equal to zero. Using the algorithm given in Problem 6.12, we can 
reduce this set to a maximal set of linearly independent vectors. Such a set must contain п vectors, 
because the subset {E,,E,,...,E,} is linearly independent; and it will contain Y, because Y is the first 
nonzero vector in the original set. [The first nonzero component is brought into the (1,1) position of the 
matrix generated by the algorithm, and it remains nonzero throughout the algorithm. Thus, the first 
vector, Y, remains part of the maximal linearly independent set.] 

Now apply the Gram-Schmidt orthogonalization process to this maximal set of linearly independent 
vectors, with Y taken as the first vector; Y remains unchanged. Finally, choose the columns of U to be 
the vectors resulting from the Gram-Schmidt process. U is unitary as a result of Problem 15.2, and the 
first column of U is the eigenvector Y. 


Apply the procedure of Problem 15.3 to construct a unitary matrix having as its first column 
an eigenvector corresponding to А = 2 for the matrix 


An eigenvector of A corresponding to A=2 is X = (0, —1, 1]", which when normalized yields 
Y = (0. - 1/V2, 1/V2]’. Applying the algorithm given in Problem 6.12 to the set consisting of Y and 


Е,=[1,0,0]7 Е, =[0,1,0]7 Е, =[0,0,1]") 


ме construct the matrix 
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This has the row-echelon form 


which indicates that the first, second, and third vectors of the set form a maximal linearly independent 
set. Applying the Gram-Schmidt process to the set {Y,E,,E,}, we obtain the orthonormal set 
(0, = Y, Q, =E,, О, =[0, 1/V2, 1/V2]"). Then 


0 1 0 
и=|-1//2 0 1/2 
1У2 0 v2 
15.5 Prove that the product of unitary matrices of the same order is also a unitary matrix. 


If A and B are unitary, then 
(АВ) ' = BA! = BA" = ВА? = (AB)' = (AB) = (AB)" 


15.6 Show that if U is unitary, then (UX, UY) = (X, Y) for all vectors X and Y of appropriate 
dimension. 


Under the Euclidean inner product, the adjoint of U is its Hermitian transpose by (13.5); hence, 
(UX, UY) = (X, U*UY) = (X, U"UY) = (X, U"'UY) = (X, IY) = (X. Y) 


15.7 Show that if A is an eigenvalue of a unitary matrix U, then |A| = 1. 
Let X be an eigenvector of U corresponding to A. Then using Problem 15.6, we have 
IA (X, X) = AX(X, X) = (AX, AX) = (UX, UX) = (X, X) a) 
Since X is an eigenvector, it is nonzero; hence (X, X) #0. Dividing (1) by (X, X), we obtain |A]? = 1, 
which implies that |A] = 1. 


15.8 Find a Schur decomposition for 


We follow Steps 15.1 through 15.5, beginning with k = 1 and T, = А. For k 2 1, А,=Т„= A. An 
eigenvalue for A is А —3, with unit eigenvector Y = (0, 1, 0]". Using the procedure given in Problem 15.3 


with n =3, we get 
010 3 1-1 

N,=|1 0 0|=0, аа T,-U/TU-|0 4 1 

001 0-1 2 


Now we apply Steps 15.1 through 15.5 with k = 2. We begin by setting 


ZEE 
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This matrix possesses the eigenvalue A=3 with corresponding unit eigenvector Y = [1/V2, -1/V2]". 
Using the procedure given in Problem 15.3 with л = 2, we generate the unitary matrix 


N -[ v2 иу? 
2 [-1У2 vv2 


which is expanded into 


| sothat Т, -U/T,U, l 3 2 
0 0 3 


z 3 22 | 


Setting U = U,U,, we have U“AU =T,, a matrix in upper triangular form. In this case, all the elements 
of U are real, so it is orthogonal. 


15.9 Find a Schur decomposition for 


We follow Steps 15.1 through 15.5, beginning with k = 1 and A, = Т, =A. An eigenvalue for A, is 
A=4, with corresponding unit eigenvector Y = (0,0, 1,0]". Using the procedure of Problem 15.3 with 


0100 4 20 2 
0010 0 30-1 
м=р o 0 0/7 апа T,=UrTU, =| 9 12 | 
0001 0-10 3 


Next we apply Steps 15.1 through 15.5 with К = 2. We first determine 


30-1 
Aj] 12 1 
-10 3 


This matrix possesses the eigenvalue A = 2 with corresponding unit eigenvector Y = (0. 1, 0]". Using ће 
procedure of Problem 15.3, now with л —3, we generate the unitary matrix 


010 
N,=|1 0 0 
001 


This is expanded into 


42 0 

nz l 1 
so that Т, = ОТО, = 00 

00 


We now apply Steps 15.1 through 15.5 with k = 3. We start by setting 


ul 3^. 

^im [ =} 3] 

This matrix possesses the eigenvalue А = 2, with corresponding unit eigenvector Y = [1/V2, 1/V2]". 
Using the procedure of Problem 15.3 with n =2, we generate the unitary matrix 
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ње inal 


which we expand into 


А 0 4 2 2/72 273 

U, =| ---—-L-------- soma — T,-U?T,U,-|O 2 0 0 
0 М2 -1У2 090: c0 М 
0 М5 М? 


Setting U = U,U,U,, we have U"AU = T,, a matrix in upper triangular form. 


15.10 Show that if U is unitary and A = U"BU, then B is normal if and only if A is normal. 
If B is normal, then B^B = BB", and 
A"A = (U"BU)"(U"BU) = (U"B"U)(U"BU) = (U"B")(UU^)(BU) 
= (U"B")(UU ' BU) = (U“B”)(BU) = U"(B"B)U = U"(BB")U 
= (U"B)(B"U) = о" воо ')(B"U) = (U^B)(UU^)(B"U) 
= (U"BU)(U"B"U) = (U"BU)(U"BU)" = AA" 
The reverse proposition is proved analogously, using the identity В = UAU”. 


15.11 Prove that every normal matrix is unitarily similar to a diagonal matrix. 
Let A be normal. Using a Schur decomposition, we can write T = U"AU, where T is upper 
triangular and U is unitary. It follows from Problem 15.10 that T is normal, and then from Problem 13.14 
that T must be diagonal. 


15.12 Prove that ||Al|,, the matrix norm induced by the Euclidean vector norm, is the square root of 
the largest eigenvalue of AYA. 

For any matrix A (not necessarily square), the product A"A is normal (Problem 13.2) and has 
nonnegative eigenvalues (Problem 13.3), which we denote as A,, А,,..., A,. It follows from Problem 
15.11 that there exists а unitary matrix U such that U"(A"A)U = D or, equivalently, such that 
АРА = UDU“, where D is a diagonal matrix whose diagonal elements are the eigenvalues of A"A. Then if 
we set А = maX(À,, A,,...,4,) and Y = U"X, we have, for any nonzero vector X, 

WAX([3 = (AX, AX) = (A*AX, X) = (A"AX, X) = (UDU"X, X) = (DU"X, U"X) = (DY, Y) 
TAX + Аў + А„у„ў, = Abl Adi e AU 
salyl Aly e А|у„| = AY. Y) = A(U"X, U"X) = A(X, UU"X) 
= A(X, X) = АХ: 


Therefore, using the result of Problem 12.14, we have 


tat, = may (Pero) = man (Prae) = va o 


Denoting as Z an eigenvector of A"A corresponding to A, we find that 
ПАД: = (AZ, AZ) = (A"AZ, Z) = (AZ, Z) = A(Z, Z) = А2]: 


MAXI). А21, | VAlIZII. 
IX], 


so IA = max ( = zh = di. 
Inequalities (1) and (2) imply ||Al|, = VA. 


EVA 2) 
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15.13 Find elementary reflectors associated with (a) У, = [1, 2]" and (5) V, 


(а) We compute ||У,|, = V5, so 
_f1 0]. 2[1 .[1 Ot aTi 2], 
r= [o 1] [з= dB al-[- 
(b) Similarly, ||У,|1, = V126, so 
100) f? 100] уга 
R,=(0 1 O|- 55] 3]{9.3,-6}=|0 1 0]|- 5| 27 
001 ~6 001 -54 
-2/7 -317 6/7 
=| -3/7 6/7 2n 
67 2n 317 
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= 19,3, -6]”. 


415 
-315 


15.14 Prove that an elementary reflector R is both symmetric and orthogonal. 
For any constant c, 
(cV V7)! = (VV7)! = ((VT)' V7 = CVV" 
Setting c = —2/||V|I2, we conclude that (-2/||V||2)VV" is symmetric. Since I is symmetric and the sum of 


real symmetric matrices is also real and symmetric, it follows that any elementary reflector is symmetric. 
In addition, 


R'R=RR=(I- WE wr )(1- ivi wj 


2 T ғ түүт 
= vvv 
= т" we’ “ТУЙ 
- VV" + vv 
tg Y * p МУУ) 


But if V is a real column vector, then УУ = (V, V) = ||V||}. Thus, the last two terms in the above 
equality cancel, R’R =I, and R is orthogonal. 


15.15 Let R be the elementary reflector associated with the vector V— X + ||Х||,Е, where X is an 


arbitrary real n-dimensional column vector, and E is an n-dimensional column vector whose 
first component is 1 and whose other components аге all 0s. Show that RX = - ||X||; E. 


Denote the first component of X as x,. Then 
VIX = (X + |XI; E)X = X"X + |[X|] E7X = Х|; + Хх, 
VIV (X + [IXIIE)'(X + [XII E) = X"X + lIXI;E"X + liXI;X"E + ([X||3E"E 
= La + жох, + Хх, + HX = 201х102 + IXa) =2У”Х 
y 2WV'X) 
2V?X 


and 


үүт 
Then ВХ = IX – 2 У" x= x= ?y vX-X- 


IE =X-V=X- (X+ {|Х|„Е) = -IXIIE 
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Supplementary Problems 
15.16 Determine which of the following matrices are unitary: 


Fi A ШУЗ из 11У 
А | ia, d B-|i1/v3 -1/V2 0 C -(1/V3) 
=2 L 0 1v3 0 -1У2 


bore 


[ivi ivi _[0У ivi 
»-[2 uei] Е ling MI 


15.17 Apply the procedure of Problem 15.3 to construct a unitary matrix having, as its first column, ап 
eigenvector corresponding to A = 3 for: 
я ыў 2 00 spi 
(ә A-[1 22] (6) в=|1 5 2] (о с=|1 51 
L1 2 J. 4 8, 
15.18 Find a Schur decomposition for each of the matrices in Problem 15.17. 


15.19 Find elementary reflectors associated with 


eB] ofo] ора 


15.20 Show that if U is unitary, then ||UX]|; = ||X||, for any vector X of suitable dimension. Thus, a unitary 
transformation preserves Euclidean length. 
15.21 The angle between two real vectors X and Y is defined as 
(X, Y) 


6 = arccos | rr 


КЕНЕ 


Show that if U is unitary, then the angle between UX and UY is the same as that between Х and Y. Thus, 
a unitary transformation preserves angles. 


15.22. Prove that В? =I for any elementary reflector R. 
15.23 Determine the eigenvalues of every elementary reflector. 
15.24 Prove that the absolute value of the determinant of a unitary matrix is 1. 
15.25 A square matrix R,,(@) is a rotation matrix if 
(1) The (р, p) and (4, 4) elements of R,, are equal to cos @ for p ¥ 4, and all other diagonal elements 
are unity, and 
(2) The (р, 4) element is equal to sin 6; the (4, p) element is equal to —sin 6, and all other off-diagonal 
elements are zero. Find 5 x 5 matrices R,,(8) and R,,(@). 


15.26 Show that a rotation matrix is orthogonal. 


15.27 Show that 0 may be chosen so that R;,(8)AR,, (0) has a zero in the (k, p) position, provided that A is 
square and k is different from p and q. (See Problem 15.25.) 


Chapter 16 


Quadratic Forms and Congruence 


QUADRATIC FORM 


A quadratic form in the real variables x,, x,,..., x, is a polynomial of the type 


5 Š ass (16.1) 


iig 
with real-valued coefficients a,,. This expression has the matrix representation 
Х'АХ (16.2) 


with A = [а, ] and X=[x,,x,,...,x,]’. The quadratic form X'AX is algebraically equivalent to 
X' ((A + A" )2)X. Since (А + A” )2i is symmetric, it is standard to use it rather than a nonsymmetric 
matrix in expression (16.2). Thus, in what follows, we shall assume that A is symmetric. (See 
Problems 16.1 and 16.2.) 

A complex quadratic form is one that has the matrix representation 


X"AX (16.3) 


with A being Hermitian. Expression (16.3) reduces to (16.2) when X and A are real-valued; and 
both expressions are equivalent to the Euclidean inner product (AX, X). 

The Euclidean inner product (AX, X) is real whenever A is Hermitian (Property 13.5). If the 
inner product is also positive (or nonnegative, negative, or nonpositive) for all nonzero vectors X, 
then the quadratic form is classified as positive definite (or positive semidefinite, negative definite, or 
negative semidefinite, respectively). All the tests listed in Chapter 14 may be applied to the matrix 
representation of a quadratic form to determine definiteness. (See Problems 16.3 and 16.4.) 


DIAGONAL FORM 


A quadratic form has diagonal form if it contains no cross-product terms; that is, if a;, = 0 for all 
іж j. It follows from Theorem 15.1 that any quadratic form can be transformed into diagonal form 
with a unitary matrix U (recall that a quadratic form is Hermitian and therefore normal). If 
U"AU = D, then the substitution X = UY converts the quadratic form (AX, X) into the diagonal 
quadratic form (DY, Y). This substitution preserves length (Problem 15.20) and angles (Problem 
15.21); the diagonal elements of D are the eigenvalues of A. (See Problems 16.5 and 16.6.) 


CONGRUENCE 


A square matrix A is congruent to a square matrix B of the same order if there exists a 
nonsingular real matrix P such that 


А = РВР” (16.4) 


When Р is factored into a product of elementary matrices corresponding to elementary row 
operations, then P" is the product in reverse order of elementary matrices corresponding to identical 
elementary column operations. Thus, two matrices are congruent if and only if one can be 
transformed to the other by a sequence of pairs of elementary row and column operations, where 
each pair consists of one elementary row operation and one elementary column operation of identical 
type. It follows that congruent matrices have the same rank. 
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A matrix A is Hermitian congruent (or conjunctive) to a matrix B if there exists a nonsingular 
matrix P such that 


A=PBP” (16.5) 


Hermitian congruence reduces to congruence when P is real. Both Hermitian congruence and 
congruence are reflexive, symmetric, and transitive. 
Two quadratic forms (AX, X) and (BY, Y) are congruent if and only if A and B are congruent. 


INERTIA 
Every n X n Hermitian matrix of rank r is congruent to a unique matrix in the partitioned form 


о 10 


(16.6) 


where I, and 1„ are identity matrices of order k x k and m X m, respectively. An inertia matrix is a 
matrix having form (16.6). 


Property 16.1: (Sylvester's law of inertia) Two Hermitian matrices are congruent if and only if 
they are congruent to the same inertia matrix, and then they both have k positive 
eigenvalues, m negative eigenvalues, and n — k — m zero eigenvalues. 


The integer k defined by form (16.6) is called the index of A, and s = k — m is called its signature. 
An algorithm for obtaining the inertia matrix of a given matrix A is the following: 


STEP 16.1: Construct the partitioned matrix [A | IJ, where 1 is an identity matrix having the same 
order as A. 

STEP 16.2: Use elementary row operations El and E3 to reduce A to upper triangular form, 
applying each operation to the full partitioned matrix of Step 16.1. In addition, 
whenever two rows are interchanged, also interchange the corresponding columns in 
the left partition, but make no similar column interchange in the right partition. 
Denote the result as [R|S], where R is upper triangular. 

STEP 16.3: Set all the nondiagonal elements of R equal to zero. The result is a partitioned matrix 
of the form [D|S], where D is diagonal. 

STEP 16.4: If a zero diagonal element of D appears іп an earlier (higher) row than a nonzero 
diagonal element of D, then interchange the positions of the two diagonal elements; 
also interchange the order of the corresponding rows of S. Continue to perform these 
interchanges until all nonzero diagonal elements of D appear in earlier rows than zero 
diagonal elements. 

STEP 16.5: І a negative diagonal element of D appears in an earlier row than a positive diagonal 
term of D, interchange the positions of the two diagonal elements; also interchange the 
order of the corresponding rows of S. Continue to perform these interchanges until all 
positive diagonal elements of D appear in earlier rows than all negative diagonal 
elements. 

STEP 16.6: И any diagonal element of the left partition is not 0, 1, or —1, denote its value by d. 
Divide that element by |d|, and divide the entire corresponding row of the right 
partition by Vd]. 


At the completion of the algorithm, the matrix in the left partition is the inertia matrix for A; the 
matrix in the right partition is the matrix P that will transform A into its inertia matrix. (See 
Problems 16.7 through 16.9.) 
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RAYLEIGH QUOTIENT 


The Rayleigh quotient for a Hermitian matrix A is the ratio 


(AX, X) 
(X.X) 


R(X) (16.7) 


Property 16.2: (Rayleigh’s principle) If the eigenvalues of A a Hermitian matrix are ordered so 


that A, € А, $7: А„, then 
A, = R(X) SA, (16.8) 


R(X) achieves its maximum when X is an eigenvector corresponding to A,; R(X) 
achieves its minimum when X is an eigenvector corresponding to A,. 


(See Problem 16.10.) 


16.1 


16.2 


Solved Problems 


Determine the symmetric-matrix representation for the real quadratic form 2x] + 5х2 + 
11х2 + 20x,x, — 4хуху + 16x2x5. 


We can rewrite this polynomial as 
духу + 5хух, + Maxx, + 10x,x, + 10x,x, = 21,5, — 2хух + 8x,x, + Вхух, 


which has the symmetric-matrix representation 
Се 2 00 -2]х, 
10 5 8]|x 
-2 8 nj» 


Determine the symmetric-matrix representation for the real quadratic form 11x? + 112 + 
19x} + 16x2 — бхух, + 10x, x, — 16x,x, — 10x,x, — lóx,x,. 
We can rewrite this polynomial as 
lxx, Ix, x, + 19хуху + 16x,x, — Эхх, 7 3x,x, + 5x x, Sx x, 7 8x,x, — BAX, — 5x4, 5x yx, 
= 8x,x, — Bx x, + 0хух, + Охх, 
which has the symmetric-matrix representation 
[х.х хха -3 5 -8][5 
= 11 -—5 es 


5 -5 19 Өлү 
-8 -8 0 J% 
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16.3 


16.4 


16.5 


16.6 


16.7 


Determine whether the quadratic form given in Problem 16.1 is positive definite. 


The results of Problems 16.1 and 14.2 indicate that the matrix representation of the quadratic form 
is not positive definite. Therefore the quadratic form itself is not positive definite. 


Determine whether the quadratic form given in Problem 16.2 is positive definite. 


From the results of Problems 16.2 and 14.3, we determine that the quadratic form is not positive 
definite because its matrix representation is not. The quadratic form is, however, positive semidefinite. 


Transform the quadratic form given in Problem 16.1 into a diagonal quadratic form. 


Given the result of Problem 16.1, we set 


A has eigenvalues —9, 9, and 18 and corresponding orthonormal eigenvectors Q, = [2/3, -2/3, 1/3], 
О, = [2/3, 1/3, -2/3]", and О, = [1/3, 2/3, 2/3]", respectively. We take 


2/3 203 13 ~9 0 0 
U=/~-2/3 13 2/3 whereupon U’AU=D=/| 0 9 0 


13 -2/3 2/3 0 0 18 


Set X - UY. Then 


X7AX = (UY)'A(UY) = Y"U'AUY = Y'DY = —9y7 + 9y2 + 18y? 


Transform into diagonal form the complex quadratic form corresponding to the Hermitian 
matrix 
з 1+ B] 
А [ 1-3 6 


The eigenvalues of A are 1 and 8, with corresponding orthonormal eigenvectors О, = [(1 + i3)/V14, 
-2/V14]’ and О, = [(1 + i3)/V 35, 5/V35]’, respectively. We set 


Га +13) М7 ФИЛЕ) Жаы © 
5-| -2/Vià siV35 Wüéreupon, "UAM. »-[; 8 


Set X = UY. Then the original quadratic form 
X"AX = (AX, X) = 3x,3, + 6x,£, + (15 i3) x, + (1 — 13)x,2, 


is transformed into the diagonal quadratic form (DY, Y) = y,y, + Ву,ў,. 


012 
A=]1 1 3 
234 


We augment the 3 x 3 identity matrix onto A and then reduce A to upper triangular form. To do so, 
we must first interchange the first and second rows, and we do this to the entire partitioned matrix. In 


Determine the inertia matrix for 
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addition, we interchange the first and second columns of A but make no corresponding change to the 


columns in the right partition. Steps 16. 


fo 1 21 0 0 
11 3/0 10 
23 4001 
fi 1 3/0 10 
6101 21 00 
23 4001 
fr т зто 
10 2/1 00 
3 2 41001 
100 зю i 
—|[0 -1 -iji -1 
3 2 40 0 
[nor зо 1 
0 -1 -ifi -1 
—-io -1 -slo -3 
? d 3] 0 
0 -1 ~| 1 
lo o -4l-1 
>f 0 до 
—-|0 -1 of 1 
0 о -4l-1 


There are no zero diagonal elements in 


ei 


1 through 16.6 are as follows: 


Interchanging the first and second 
rows 


Interchanging the first and second 
columns of the left partition only 


0 Adding —1 times the first row to 
0 the second row 
1 
0 Adding —3 times the first row to 
0 the third row 
1 
10 Adding —1 times the second row 
0 to the third row 
=2 4 
10 Setting all the elements above the 
0 diagonal equal to zero in the left 
-2 1 partition 


the left partition, so Step 16.4 is satisfied. Also, the positive 


diagonal element in the left partition appears in an earlier row than the negative elements, so Step 16.5 


is satisfied. However, the third row has a 


diagonal element in its left partition that is not equal to 0, 1, or 


= 1; following Step 16.6, we divide that diagonal element by |d| = 4, and the remainder of the third row 


by V4 = 2. This gives us 


1 
=i 
-1 


so 


The index of A is 1, and its signature is 1 


о 0| 0 1 «| 
=} 0| 1 m 0 
0 -1l-1/2 -1 1/2 
0 1 0 0 
«| and me" [0 -1 e] 
1/2 0 0-1 


—2 = - 1. We may conclude from Property 16.1 that A has one 


positive eigenvalue, two negative eigenvalues, and no zero eigenvalues. 


16.8 Determine the inertia matrix for 


NANN 
eu a0 
онә 
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16.9 
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Augmenting onto A the 4 x 4 identity matrix, and then reducing A to upper triangular form, опе 

column at a time and without using elementary row operation E2, we finally obtain 

1 ooo 

0 100 

0 90 0 0|-*1-11 0 

0 201 
The left partition is in upper triangular form. Setting all elements above the main diagonal in that 
partition equal to zero yields 


1 00 0 1 ооо 
0-2 0 O| -2 100 
0 00 01-1-1110 
0 0 0 161-3 01 


Following Step 16.4, we interchange the diagonal elements in the third and fourth rows of the left 
partition while simultaneously interchanging the entire third and fourth rows of the right partition. The 
result is 


econ 
оооо 
П 
ө 
-n= 


Following Step 16.5, we next interchange the (2,2) diagonal element with the (3,3) diagonal element in 
the left partition and simultaneously interchange the order of the second and third rows in the right 
partition. That gives us the partitioned matrix 


10 00 1 000 
016 00]|-3 201 
0 0 -2 0) -2 100 
0: O ЙЕ Pb 


Following Step 16.6, we divide the (2,2) element of the left partition by 16, and the entire second row of 
the right partition by 4. We also divide the (3,3) element of the left partition by 2, and the entire third 
row of the right partition by V2. We get, finally, 


10 00 1 0 0 0 
01 00] -34 12 0 1⁄4 
00-10[|-2v2 1/20 0 
оо oot -1 -110 
1 0 0 0 10 00 
-3/4 12 0 14 7101 00 
и BEIN о 0 | 20 РАР ү 
-1 о- 1 0 00 00 


The index of A is 2, and its signature is 2—1=1. The rank of A is 3, and we may conclude from 
Property 16.1 that A has two positive eigenvalues, one negative eigenvalue, and one zero eigenvalue. 


Discuss the rationale of the algorithm given by Steps 16.1 through 16.6 as it pertains to real 
matrices. 


Since A is symmetric, any set of elementary row operations of the first and third kind that reduce A 
to upper triangular form yields an analogous set of column operations that will reduce A to lower 
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triangular form. Under a congruence transformation, both sets of operations are applied to A, resulting 
in a diagonal matrix. This is the rationale for Steps 16.1 through 16.3. 

Interchanging the position of two diagonal elements of a diagonal matrix is equivalent to 
interchanging both the rows and the columns in which the two diagonal elements appear. We 
interchange only the designated rows in P, since a postmultiplication by P^ will effect the same type of 
column interchange automatically. This is the rationale for Steps 16.4 and 16.5. 

Finally, a nonzero diagonal element d is made equal to 1 in absolute value by dividing its row and its 
column by УД]. Since the divisions will be done in tandem, we have Step 16.6. 


16.10 Prove Rayleigh’s principle. 


Let U be a unitary matrix that diagonalizes A. Then 


U”AU = 


and we may assume that the columns of U have been ordered so that A, = A, =: · = А„. Setting X = UY 
and using Property 15.3, we have 


(АХ, X) _ (AUY, UY)  (U"AUY, Y) = (DY. Y) 


R(X) = 


(X,x) (Uy, Uy) — (¥, ¥) (v, Y) 
Му + Ау? +++. + Ау? АУРА 
- = 

Ту Hy Т, [у + lyf + 


The other inequality follows from 


Мә + Aal yal? +++ + Ау, S A du + Adi et Ау? 
If X, is an eigenvector corresponding to A,, then 


R(X,) = зуу" Чык SL mA, 


so the minimum of R(X) is achieved when X = X,. A similar argument shows that the maximum is 
achieved when X is an eigenvector corresponding to А,. 


Supplementary Problems 


16.11 Determine symmetric matrix representations for the following real quadratic forms: 
(а) Зх? + 3x} + Sx} + 2x,x, 7 2x,x, —2хух, 
(b) 2x1 + 2x3 + 5х2 + Ax x, 7 23,3, 7 2x,x, 
(с) 9x1 + 6x3 + 9x3 + 6x2 — 6x x, — бхух, + 6x, x, — бхух, 
(d) xi 3x1 + 333 + x1 - 2x, x, + Axx, — 2x x, 8x24, + Axyx, + 2хух, 


16.12 Determine which of the quadratic forms in Problem 16.11 are positive definite. 


16.13 Determine the inertia matrix associated with each of the quadratic forms in Problem 16.11. 
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16.14 


16.15 


16.16 


16.17 


16.18 


16.19 


16.20 


16.21 


16.22 


Using the results of Problem 16.13, determine whether quadratic forms (a) and (b) of Problem 16.11 are 
congruent. 


Using the results of Problem 16.13, determine how many positive and negative eigenvalues are 
associated with the symmetric matrix corresponding to each quadratic form in Problem 16.11. 


Characterize the inertia matrix of a positive definite quadratic form 


Find a nonsingular matrix P such that PAP" is an inertia matrix for 


0 2 4 42 3 
(a) [ 1 i (b) p 4 d 
4-2 $ 365 


Determine the inertia matrix associated with 


1 1-i2 i 
А=|1+2 6 3*0 
me d 2 


and find P such that PAP" is that inertia matrix. 
Show that if A is congruent to B and B is congruent to C then A is congruent to C. 
Show that if A is congruent to B, then B is congruent to A. 


Prove that two Hermitian matrices A and B are congruent to each other if and only if they are congruent 
to the same inertia matrix. 


Show that a nonsingular Hermitian matrix A is congruent to its own inverse. 


Chapter 17 


Nonnegative Matrices 


EIGENVALUES AND EIGENVECTORS 


A matrix A is nonnegative, written А 2 0, if all its elements are real and nonnegative; A is 
positive, written A > 0, if all its elements are real and positive. A matrix A is greater than a matrix В 
of identical order, denoted A> B, if A — B is positive. Similarly, А =B if A — B is nonnegative. 

The spectral radii (see Chapter 12) of nonnegative square matrices have the following properties: 


Property 17.1: If 0<A<B, then о(А) = o(B). 
Property 17.2: If A 20 and if the row (or column) sums of A are a constant К, then o(A) = k. 


Property 17.3: If т is the minimum of the row (or column) sums of A, M is the maximum of the 
row (or column) sums of A, and А «0, then m = o(A) = M. 


Property 17.4: А nonnegative square matrix has an eigenvalue equal to its spectral radius, and 
there exist a right eigenvector and a left eigenvector corresponding to this eigenvalue 
that have only nonnegative components. 


Property 17.5: (Perron’s theorem) A positive square matrix has an eigenvalue of multiplicity one 
equal to its spectral radius, and no other eigenvalue is as large in absolute value. 
Moreover, there exist a right eigenvector and a lett eigenvector corresponding to 
c (A) that have only positive components. 


(See Problems 17.1 to 17.6.) 


IRREDUCIBLE MATRICES 


A permutation matrix is a matrix obtained from an identity matrix by any rearrangement of its 
rows. Such a matrix is the product of elementary matrices of the first kind and is orthogonal. A 
nonnegative matrix is reducible if there exists a permutation matrix P such that 


(17.1) 


where both A,, and A,, are square matrices having order less than that of A. If no such permutation 
matrix exists, then A is said to be irreducible. (See Problems 17.7 and 17.8.) 


Property 17.6: Positive matrices are irreducible. 
Property 17.7: An n X n matrix A is irreducible if and only if (1 + A)! 29. 


Property 17.8: (Perron-Frobenius theorem) A nonnegative, irreducible matrix A has an eigenvalue 
of multiplicity one equal to its spectral radius, and corresponding to this eigenvalue 
is a right (left) eigenvector which has only positive components. If such a matrix has 
exactly k eigenvalues with absolute value equal to its spectral radius, then they are 
of the form w,o(A), where w,, w;,.... W, are the k distinct roots of unity. 


(See Problems 17.9 to 17.11.) 
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PRIMITIVE MATRICES 


A nonnegative matrix is primitive if it is irreducible and has only one eigenvalue with absolute 
value equal to its spectral radius. A nonnegative matrix is regular if one of its powers is a positive 
matrix. A nonnegative matrix is primitive if and only if it is regular. 


Property 17.9: If A is a nonnegative primitive matrix, then the limit L=lim,,_... ({1/o(A)}A)” 
exists and is positive. Furthermore, if X and Y are, respectively, left and right 
positive eigenvectors of A corresponding to the eigenvalue equal to (A) and scaled 
so that YX=1, then L= XY. 


Positive matrices are primitive and have the limit described in Property 17.9. (See Problem 17.12.) 
Reducible matrices may or may not have such a limit. (See Problem 17.13.) 


STOCHASTIC MATRICES 


A nonnegative matrix is stochastic if all its row sums or all its column sums equal 1. It is doubly 
stochastic if all its row sums and all its column sums equal 1. It follows from Property 17.2 that the 
spectral radius of such a matrix is unity. If the row (column) sums are all 1, then a right (left) 
eigenvector corresponding to А = 1 has all its components equal. 

A stochastic matrix is ergodic if the only eigenvalue of absolute value 1 is 1 itself, and if the 
eigenvalue А = 1 has multiplicity К, then there exist k linearly independent eigenvectors correspond- 
ing to it. 

Property 17.10: If P is ergodic, then lim,,_... P" = L exists. 


A primitive stochastic matrix A is ergodic with К = 1 and has a simple form for the limiting 
matrix L. If the row (column) sums of A are all 1, then the same is true for the row (column) sums of 
L, and all the rows (columns) of L are identical. Each of these rows (columns) is the unique left 
(right) eigenvector corresponding to A = 1 and having the sum of its components equal to unity. (See 
Problems 17.14 and 17.19.) 

The form of the limiting matrix is not as simple for an ergodic matrix that is not primitive. A 
canonical basis for such a matrix consists solely of eigenvectors. If the multiplicity of A = 1 is denoted 
by К, and if the k linearly independent right eigenvectors corresponding to this eigenvalue are placed 
into the first k columns of the modal matrix M, then L = МОМ ', where D is a diagonal matrix 
having its first k diagonal elements equal to unity and all others equal to zero. (See Problem 17.15.) 


FINITE MARKOV CHAINS 


An N-state Markov chain consists of a set of objects and a finite set of N different states (where 
N is a fixed positive integer), such that (1) at any given time each object is in one of the N states, 
which may be different for different objects, and (2) the probability that an object will move from 
one state to another state (or remain in the same state) in one time period depends only on the 
beginning and ending states. The N X № matrix P — [р,], where p, denotes the probability of an 
object moving from state i to state j іп one time period, is stochastic. The (i, j) element of the mth 
power of P represents the probability that an object will move from state i to state j in m time 
periods. 

Denote the proportion of objects in state i at the end of the mth time period as x”, and define 


xe =], a0... x0] 
to be the distribution vector for the end of the mth time period. Then 


(0) сү) (0) (0) 
Xx" pay key are i^] 


154 NONNEGATIVE MATRICES (CHAP. 17 


represents the proportion of objects in each state at the beginning of the process. Necessarily, 
x > 9, and the sum of the components of X“ is 1 for each m=0,1,2,.... Furthermore, 


x"! = xp" (17.2) 
If P is primitive, then 

XC? = lim X? = ХӘБ (17.3) 
which is the positive left eigenvector of P corresponding to A=1 and having the sum of its 
components equal to unity. The ith component of x represents the approximate proportion of 
objects in state i after a large number of time periods, and this limiting value is independent of the 
initial distribution defined by X. If P is ergodic but not primitive, (17.3) still may be used to obtain 
the limiting state distribution, but it will depend on the value of X. (See Problems 17.16 and 

17.17.) 


Solved Problems 


17.1 Estimate the location of the largest eigenvalue of 


A is nonnegative, and its row sums are 7, 16, and 5. It follows, then, from Property 17.3, that 
5< о(А) = 16. However, the column sums of A are 3, 11, and 14, so 3= o(A) = 14. Together, the two 
inequalities imply that the largest eigenvalue is between 5 and 14 in absolute value. 


17.2 Verify Property 17.4 for 


The eigenvalues of A are A, = A; 7 1 and A, = —1; and, since о(А) = 1 by definition, there is an 
eigenvalue equal to the spectral radius. A right eigenvector corresponding to А = 1 is (0, 1, 1)’, which is 
nonnegative. A left eigenvector corresponding to the same eigenvalue is [1,0,0], which is also 
nonnegative. 


17.3 Verify Perron's theorem for 


0.1 0.5 0.4 
A=|0.7 02 0. 
0.6 0.2 0. 
The row sums of A are all equal to 1, so it follows from Property 17.2 that o(A)=1. The 
characteristic equation of A is d(A)=—A?+0.5A? + 0.53A — 0.03, which has the roots 4,71, А, = 
—0.554138, and A, = 0.054138. Thus, the spectral radius is an eigenvalue of mult ity one, and it is the 
greatest eigenvalue in absolute value. A right eigenvector corresponding to this eigenvalue is (1, 1, 1”, 
while a left eigenvector is [62, 48, 37]. Each of these has only positive components. 


17.4 Prove that if 0<А <В, where A and B are square matrices of the same order, then 
о(А) = a(B). 
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17.5 


17.6 


17.7 


17.8 


If 05 A <В, then A" s B" for any positive integer т and, therefore, |А" ||, = ||B"||,. It follows 
from (12.4) that 


(A) = lim А” = lim |В" = o(B) 


Prove that if the row (or column) sums of a nonnegative square matrix A are a constant k, 
then o(A) = k. 


Using (12.3), we may write 
o(A) = |А. = k a) 


If we set X=[1,1,...,1]’, it follows from the row sums being k that AX = kX, so that k is an 
eigenvalue of A. Since o(A) must be the largest eigenvalue in absolute value, 


о(А)=к (2) 
Together, (1) and (2) imply с(А) = k. The proof for column sums follows if we consider AT in place of 


Prove that if m is the minimum row (or column) sum and M is the maximum row (or column) 
sum of an n X n nonnegative matrix А =(a,], then т = о(А) = M. 


Construct a matrix B = [b] having the same order as A and such that 


0 ifm=0 


The A =B z 0, and the row sums of B are all equal to т. It follows from Problem 17.4 that о(В) = о(А) 
and from Problem 17.5 that o(B) = m; thus, m = o(A). That M is the upper bound on c(A) follows from 
(12.3); that is, (А) = ||А|| „ = M. The analogous result for column sums is obtained through an identical 


argument applied to A’, 


Determine whether the matrix in Problem 17.1 is irreducible. 


The matrix possesses a single zero element, and it is on the main diagonal. An elementary row 
operation of the first kind (E1) followed by an elementary column operation of the same kind will 
change the positions of diagonal elements, but not their values. Since a permutation matrix P is a 
product of elementary matrices of the first kind, it follows that PAP” also leaves the values of diagonal 
elements unchanged. Such a transformation cannot move the zero into the (3,1) position and cannot 
result in additional zero elements. Since there are no other zeros available to move into that position, 
the matrix is irreducible. 


Determine whether the matrix in Problem 17.2 is irreducible. 
An interchange of the first and third rows followed by an interchange of the first and third columns 


gives us the transformation 
00 ıı 0 оўо O 1 013 
=|0 10/2 © 110 1 0|=[10 2 
10 O13 1 011 0 0 001 


which has the partitioned form given in (17.1) with 


РА! 
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Therefore, A is reducible. 


Determine whether the following matrix is irreducible: 


0200 

100 4 0 

А ооо 2 

1000 

To use Property 17.7, we calculate 

1 6 24 16 
; [8 1 12 24 
А+зу= 1 6 
36 8 1 


Since this matrix is positive, A is irreducible. 


Verify the Perron-Frobenius theorem for the matrix in Problem 17.1. 


The matrix is irreducible (see Problem 17.7), and its eigenvalues may be found, to four decimal 
places, to be 10.1806, —1.5631, and 1.3825. Its spectral radius is thus 10.1806, which falls within the 
bounds identified in Problem 17.1. A right eigenvector corresponding to this spectral radius, and with all 
components rounded to four decimal places, is (0.2611, 1, 0.2764]". A left eigenvector, rounded 
similarly, is (0.2893, 1, 0.9456]. Both eigenvectors have only positive components. 


Verify the Perron-Frobenius Theorem for the matrix in Problem 17.9. 


The characteristic equation of А is d(A) = A* — 16, so its eigenvalues are 2, —2, i2, and —i2, all of 
which have absolute value equal to the spectral radius of 2. Each eigenvalue is the product of the 
spectral radius and one of the four fourth roots of unity, 1, —1, i, and —i. A right eigenvector 
corresponding to A = 2 is [2, 2, 1, 1]’, while a left eigenvector is [1, 1, 2, 2]; both are positive. 


Determine whether the matrices given in Problems 17.1, 17.2, 17.3, and 17.9 are primitive. 


The matrix A in Problem 17.1 is irreducible (Problem 17.7) and has only one eigenvalue with 
absolute value equal to its spectral radius (Problem 17.10), so it is primitive. Alternatively, А? is 
positive, so A is regular and, therefore, primitive. 

The matrix in Problem 17.2 is reducible (Problem 17.8) and, therefore, cannot be primitive. 

The matrix in Problem 17.3 is positive and, therefore, primitive. 

The matrix in Problem 17.9 is irreducible, but it has four eigenvalues having absolute value equal to 
its spectral radius (Problem 17.11), so it is not primitive. Alternatively, one can show that the (1,1) 
element of every power of A is zero, so A is not regular and, therefore, not primitive. 


Show that a square matrix A need not be primitive to possess the limit L= 
lim, ..((1/0(A)) A)". 


The matrix 


^ 


has spectral radius o(A) = 2. For this matrix, ({1/o(A)}A)” =I for every positive integer m, so L= I. 
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17.14 


17.15 


17.16 


Find L = lim,, ., A" for the matrix in Problem 17.3. 


The matrix is stochastic and primitive (Problem 17.12), and it has a left eigenvector given by 
[62, 48, 37]. If we divide each component of that eigenvector by the sum of the components, 62 + 48 + 
37 = 147, we obtain a positive left eigenvector whose components sum to unity. Then 


62/147 48/147 37/147 
L={ 62/147 48/147 37/147 
62/147 48/147 37/147 


Determine whether the stochastic matrix 
1 0 0 0 
p-|04 0 06 0 
02 0 01 0.7 
ооо 1 
is ergodic, and, if so, calculate L=lim,,_.. P". 
The eigenvalues of P are A, = A, = 1, A, =0.1, and A, =0, so the matrix is not primitive. P does, 
however, possess two linearly independent right eigenvectors corresponding to А = 1, 
[45.24, 10,0" апа — [~35, - 14,0, 10]" 
so it is ergodic and L exists. As an easy calculation shows, the right eigenvectors 
[0,6,1,0}7 and —[0,1,0,0]" 
correspond, respectively, to A, and A,. Thus, 
45 -35 0 0 1/45 O 0 7/90 
_|24 -14 6 1 ATE 0 0 1/10 
М” ол | 20 М"= зо 1 -7/9 
0 1000 4/5 1 -6 21/5 
45 -35 0 011 0 O 071/45 0 0 7/90 1.00 0 
d L-| 74 -14 6 11/0 100 о 0 O 1/10) |8/15 0 O 7/15 
Ан =jio o 1 Off0 o o off-z9 0 1 -79|7|29 0 0 7/9 
о 10 0 010 O O ОД 4/5 1 -6 21/5 0 00 1 
Formulate the following problem as a Markov chain and solve it: New owners of rental 


apartments in Atlanta are considering as the operating agent a real estate management firm 
with a reputation for improving the condition of antiquated housing under its control. Based 
On initial ratings of poor, average, and excellent for the condition of rental units, it has been 
documented that 10 percent of all apartments that begin a year in poor condition remain in 
poor condition at the end of the year, 50 percent are upgraded to average, and the remaining 
40 percent are renovated into excellent condition. Of all apartments that begin a year in 
average condition, 70 percent deteriorate into poor condition by year’s end, 20 percent remain 
in average condition, while 10 percent are upgraded to excellent. Of all apartments that begin 
a year in excellent condition, 60 percent deteriorate into poor condition by the end of the 
year, 20 percent are downgraded to average, and 20 percent retain their excellent rating. 
Assuming that these findings will apply if the firm is hired, determine the condition that the 
new owners can expect for their apartments over the long run. 


We take state 1 to be a rating of poor, state 2 to be a rating of average, and state 3 to be a rating of 
excellent. With percentages converted into their decimal equivalents, the probabilities of moving from 
State i to state j (i, j = 1,2, 3) over a one-year period are given by the elements of the stochastic matrix 
in Problem 17.3. Using the results of Problem 17.14 and Eq. (17.3), we have 
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X'? = (62/147, 48/147, 37/147] = (0.422, 0.327, 0.252] 


Over the long run, approximately 42 percent of the apartments controlled by this real estate manage- 
ment firm will be in poor condition, 33 percent will be in average condition, and 25 percent will be in 
excellent condition. 


Formulate the following problem as a Markov chain and solve it: The training program for 
production supervisors at a particular company consists of two phases. Phase 1, which involves 
three weeks of classroom work, is followed by phase 2, which is a three-week apprenticeship 
program under the direction of working supervisors. From past experience, the company 
expects only 60 percent of those beginning classroom training to be graduated into the 
apprenticeship phase, with the remaining 40 percent dropped completely from the training 
program. Of those who make it to the apprenticeship phase, 70 percent are graduated as 
supervisors, 10 percent are asked to repeat the second phase, and 20 percent are dropped 
completely from the program. How many supervisors can the company expect from its current 
training program if it has 45 people in the classroom phase and 21 people in the apprenticeship 
phase? 


We consider one time period to be three weeks, and define states 1 through 4 as the classification of 
being dropped, a classroom trainee, an apprentice, and a supervisor, respectively. If we assume that 
discharged individuals never reenter the training program and that supervisors remain supervisors, then 
the probabilities of moving from one state to the next are given by the stoch matrix in Problem 
17.15. There are 45 + 21 = 66 people currently in the training program, so the initial probability vector 
for current trainees is X? = [0, 45/66, 21/66, 0]. It follows from Eq. (17.3) and the results of Problem 
17.15 that 


1 00 0 

хер = 8/15 0 0 7/15|_ 

X -X'"L-(0,45/66,21/66,0] 57 о p 7/9 |= 10-4343, 0, 0, 0.5657) 
0001 


Eventually, 43.43 percent of those currently in training (or about 29 people) will be dropped from the 
program, and the rest (about 37 people) will move into supervisory positions. 


Prove that the product of two stochastic matrices, both of which have their row (or column) 
sums equal to unity, is itself a stochastic matrix of the same type. 


Let A and B denote stochastic matrices with row sums equal to unity, and set C = AB. Then the ith 
row sum of C is 


Prove that if P is a primitive stochastic matrix with row sums equal to unity, then all the rows 
of L = їтї„_.„ Р” are identical. 


It follows directly from Problem 17.18 that if P is an n X n stochastic matrix, then so too is any 
positive integral power of P. Therefore, L is necessarily stochastic too. Now 
L= lim P" = lim (P”-'P) = (lim P"^')P- LP 


which implies that every row of L is a left eigenvector of P corresponding to A — 1. Since P is primitive, 
A=1 is an eigenvalue of multiplicity one, and all eigenvectors corresponding to it must be scalar 
multiples of each other. With L being stochastic, the row sums of L must all be unity, so it follows that 
the rows are identical. 
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Supplementary Problems 


In Problems 17.20 through 17.29, determine whether the given matrix is irreducible, primitive, or 
stochastic, and estimate its spectral radius. For those matrices P that are stochastic, determine lim,, ,. Р” if it 
exists. 


120 121 020 
1720 |2 2 0 17.21 101 17.22 001 
112 2 2. 100 
121 г H 1 3 0.1 08 0.1 
17.23 |1 1 2 17.24 1402 1725 |09 0 01 
211 2241 0.2 02 06 
1 0 0 05 0 0.5 100 
17.26 |0.21 0.79 0 17.27 0 I 0 17.28 001 
0.17 0.35 0.48 03 0 07 010 
01 0.6 0.3 
17.29 |0.6 0.2 02 
03 02 05 


17.30 The manufacturer of Hi-Glo toothpaste currently controls 60 percent of the market in a particular city. 
Data from the previous year show that 88 percent of Hi-Glo's customers remained loyal to Hi-Glo, while 
12 percent switched to rival brands. In addition, 85 percent of the competition's customers did not switch 
to Hi-Glo during the year, while the other 15 percent did. Assuming that these trends continue, 
determine Hi-Glo's share of the market (a) in 5 years and (b) over the long run. 


17.31 Grape harvests in the Sonoma Valley are classified as either superior, average, or poor. Following а 
superior harvest, the probabilities of having superior, average, and poor harvests the next year are 0, 
0.8, and 0.2, respectively. Following an average harvest, these probabilities are 0.2, 0.6, and 0.2, 
respectively; following a poor harvest, they are 0.1, 0.8, and 0.1, respectively. Determine the probability 
of a superior harvest for each of the next 5 years, if the most recent harvest was average. 


17.32 The geriatric ward of a hospital lists its patients as bedridden or ambulatory. Historical data indicate that 
over a one-week period, 30 percent of all ambulatory patients are discharged, 40 percent remain 
ambulatory, and 30 percent are remanded to complete bed rest. During the same period, 50 percent of 
all bedridden patients become ambulatory, 20 percent remain bedridden, and 30 percent die. Currently 
the hospital has 100 patients in its geriatric ward, with 30 bedridden and 70 ambulatory. Determine the 
expected status of these patients (a) after 2 weeks, and (b) over the long run. (The status of a discharged 
patient does not change if that patient later dies away from the hospital.) 


Chapter 18 


Patterned Matrices 


CIRCULANT MATRICES 


A circulant matrix is a square matrix in which every row beginning with the second can be 
obtained from the preceding row by moving each of its clements one column to the right, with the 
last element circling to become the first. Circulant matriccs have the general form 


а а, ay a, | 

а, а а, lisa 
A=|4,1 а, а, 4,2 

а ау à, а, 


Note that each diagonal consists of identical clements. (Sec Problem 18.1.) 
Property 18.1: If à circulant matrix A has order л x n, then its eigenvalues are 
Aja ar + ау tear’) (i=1,2,...,n) 


where [4,. à;,. .. , a, | is the first row of A and r, is onc of the n distinct solutions of 


r"=1, The corresponding eigenvectors are X, = [1, r; 7,..., 77]. (See Prob- 
lems 18.2 and 18.3.) 


Property 18.2: If A and B arc circulant matrices of thc same order and a and b are any two scalars, 
then аА + БВ is also a circulant matrix. 


Property 18.3: The product of two circulant matrices of the same order is itself a circulant matrix, 
and the product is commutative. 


Property 18.4: If a circulant matrix is nonsingular, then its inverse is a circulant matrix. 


BAND MATRICES 


A square matrix А = [a;] having order n X n is a band matrix of width 2K + 1 if а, = 0 when 
|i — j|» К for some nonnegative integer К between 0 and n — 1. In а band matrix, all nonzero 
elements are positioned on the main diagonal and the first K diagonals directly above and below the 
main diagonal, The general form of such a matrix is 


A diagonal matrix is a band matrix with К = 0. 


Property 18.5: The sums, products, and transposes of n X n band matrices of width 2K +1 are 
band matrices of the same width. 


A Toeplitz matrix is a band matrix in which each diagonal consists of identical elements, although 
different diagonals may contain different elements. Every nonzero circulant matrix is a Toeplitz 
matrix of full width, 
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TRIDIAGONAL MATRICES 


A tridiagonal matrix is a band matrix of width three. Nonzero elements appear only on the main 
diagonal, the superdiagonal, and the subdiagonal; all other diagonals contain only zero elements. 


Property 18.6: The eigenvalues of an n X n tridiagonal Toeplitz matrix with elements a on the main 
diagonal, b on the superdiagonal, and c on the subdiagonal are 


Ay = а + 2Vbccos > (к=1,2,..., п) 


Кт 
nt 
(See Problem 18.4.) 

Crout’s reduction (see Chapter 3) is an algorithm for obtaining an LU factoriza- 
tion of a square matrix such that L = [/,] is lower triangular, and U = [u,,] is upper 
triangular with unity elements on the main diagonal. For a tridiagonal matrix 
А = [a;] of order n x n, the algorithm simplifies to the following: 


STEP 18.1: Initialization: If a,, = 0, stop; factorization is not possible. Otherwise, set /,, = @,,; set 
the subdiagonal of L equal to the subdiagonal of A; set each diagonal element of U 
equal to unity; set all other elements of L and U equal to zero; and set a counter at 


і=2. 

STEP 18.2: Calculate u;_, аа-а 

STEP 18.3: Calculate l, =a, — 1, 1ш: If i= п, stop; the algorithm is complete. 

STEP 18.4: If l, is zero, stop; factorization is not possible. Otherwise, increase i by 1 and return to 
Step 18.2. 


This factorization will produce an L matrix having nonzero elements only on its diagonal and 
subdiagonal, and a U matrix having nonzero elements only on its diagonal and superdiagonal. (See 
Problems 18.5 and 18.6.) 


HESSENBERG FORM 


A square matrix is in Hessenberg form if all elements below the subdiagonal are zero. Every real 
square matrix A is congruent (see Chapter 16) to a matrix in Hessenberg form. An iterative 
algorithm for effecting this transformation successively generates, at each stage, matrices A, and P, 
(= 1,2,...,п- 2), where P, is orthogonal and A, has its first k columns in Hessenberg format. 
Then P=P,P,---P,_, and Р'АР=Н, where Н is in Hessenberg form. Since P is orthogonal, 
А = PHP . For notational convenience, we set Ay = А. The kth iteration of the algorithm is: 


STEP 18.5: Set X, equal to the kth column of А, |, restricted to those elements that are below the 
main diagonal. Thus, X, contains n — k components. 

STEP 18.6: Construct V, =X, + ||X,||;E,, where E, is an (л — k)-dimensional column vector 
having its first component equal to 1 and all other components equal to zero. 

STEP 18.7: Determine the elementary reflector R, associated with V,, using (15.4). 


STEP 18.8: Set 
„|0 
„= 


where I, denotes the К x k identity matrix, and calculate A, = PA, ,P,. 


If A is real symmetric, then the resulting matrix in Hessenberg form is also real symmetric and, 
therefore, tridiagonal. (See Problems 18.7 and 18.8.) 
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Solved Problems 


[CHAP. 18 


18.1 Determine whether the following matrices are circulant, Toeplitz, band matrices, tridiagonal, 


and/or in Hessenberg form: 


2-1 0 0 
ї 3-4 12-1 0 
J dd = 3 do i 2-1 
AA o-4- 4-2 В= о; 0 1 2 
- 3-4 1 0 0 0 1 
0 о 0 0 
12-1 0 0 0 1 2959 
21330 
01 2-1 0 0 21259 
c- 00 1 2-1 0| p= 
00 0 1 2-1 „ЖА e. 
-10 00 1 2 оосо 2 
00002 
12 0 00 12345 
-1 2 -2 00 51234 
Е-| 02 1 10 F=|05123 
00 3-11 00512 
00 0-32 00051 


A is a circulant matrix and a Toeplitz mat 
B is a band matrix of width 3, tridiagonal, 
C is a circulant matrix. 

D is a band matrix of width 5; it is in Hessenberg form. 

E is a band matrix of width 3, tridiagonal, and in Hessenberg form. 
F is a Toeplitz matrix in Hessenberg form. 


Momooo 


it is a band matrix of full width which, in this case, is 7. 
nd in Hessenberg form; it is also a Toeplitz matrix. 


18.2 Determine the eigenvalues of, and a canonical basis for, matrix A in Problem 18.1. 


A is a circulant matrix having order 4 x 4. The roots of r“ = 1 are r, = 1, r, = —1, r, =i, 7% = —1,50 


A, = 1+ (=2)(1) + 3(1)? + (-4)(1)? = -2 with =X, -[L 1,1, 1]" 
A, 1+ (—2)(=1) + 30-1)? + (74)(7 0? = 10 with Х,=[1,-1,1,-1]' 
Ay = 1+ (—2)(i) + 3G) + (-4)) = -2 + i2 with — X,-2[Là -1,-i]" 


A,71*(-2)(7i) +3(~i} + (-4)(-)° = -2- i2 with ХХ, = [1,4 71, i]" 


18.3 Derive Property 18.1. 


Denote the elements in the first row of a circulant matrix A as а,,а,,... 
let r be any root of 


r=l 


Set ya, tartay +a, rm + a,r" 
Multiplying (2) successively by 7, 7°, . . . ,7”~' gives the system of equations 
yr=a,+a,r +a, + 

yeaa, tartar +: 


„а, from left to right, and 


а) 
(2) 
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This system, with (2), has the matrix form yX = AX, for X = [1, r, 7^... , 7" ']". Thus, y, as given by 
(2), is an eigenvalue, and X is a corresponding eigenvector for every root r. 

Given (2) and the fact that г = 1 is always a root of (1), it follows that the sum of any row of a 
circulant matrix is an eigenvalue of that matrix. 


18.4 Determine the eigenvalue of matrix В in Problem 18.1. 
Using Property 18.6 with a —2, b = —1, and c = 1, we have A, = 2 + 2V(-1)(1) cos (km/7), which, 
for k 2 1,2,. ., 6, yields 
Ay = 2+ i1.801938 А,=2 + 11.246980 А, = 2 + 10.445042 
A, = 2 – i0.445042 Ay = 2 – i1.246980 Ag = 2 – i1.801938 


18.5 Determine an LU decomposition for matrix B in Problem 18.1. 


We apply Steps 18.1 through 18.4 to B = [b, ], initializing to 


2000020 1000020 
100000 010000 
1-19 19000; ,.|[001000 
001000 000100 
000100 000010 
0000190, 000001 
and then calculating as follows: 
; из = bylh, =—1/2 
хоад log = ba ша 927 71/2) = 5/2 
И uy = ball; = -1/(5/2) = -2/5 
For і= 3: Ds = by — bati = 2 — I(-2/5) = 12/5 
| uy, = bylly = —1/(12/5) = -5/12 
For і = 4: Dag — [uy = 2— 1(75/12) = 29/12 
- Uys = Б, = 71/(29/12) = ~12/29 
For i= 5: dss = bss latas = 2 — 1(-12/29) = 70/29 
І Use = balls, = —1(70/29) = –29/70 
For і = 6: lss = bu — = 2- 1/(—29/70) = 169/70 


The factorization is, then, 


2 о 0 0 0 0 1 -12 б 0 0 0 

152 0 0 0 0 0. d -—2/5 0 0 0 
в=|0 1 125 0 0 0 0 0 1 -5/12 0 0 

о 0 1 29/12 0 0 0 0 0 1 —12/29 0 

0 0 0 1 70/29 0 0 0 0 0 1 —29/70 

0 0 0 0 1  169/0]lo0 0 0 0 0 1 


18.6 Determine an LU decomposition for matrix E in Problem 18.1. 


We apply Steps 18.1 through 18.4 to E = [е,], initializing to 


100 00 10000 
-100 00 01000 
L-|020 00 u=|0 0100 
003 00 00010 
000 -30 00001 


and then calculating: 
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из = enll, 21272 

ИГЕ 127 еш 

= baa = еа lau =2-(-1)(2)=4 
из = enll = 72/47 -1/2 

keien аз = ёз 

nix lyy = 6,7 lus = 1 - 2(-1/2) 22 

NU Uy, = €/l, = 1/2 
For i24: lames luus 71-730/2)7 -5/2 
— иш = е = 1/(—5/2) = -215 


lys mes 7 luus 72- (-3)(-2/5) = 4/5 
The factorization is, then, 


100 0 0112 0 0 0 

-140 0 O01 -12 0 0 

E=| 022 о ооо 1 12 0 
003 -52 000 0 1 =215 

000 -3 4/5100 0 0 1 


21-2 

E 1 d 

43 1 

The first iteration (k = 1) of Steps 18.5 through 18.8 yields 


x-[1] 
мех, - [73] s (1]-(2] 


берг t Ge e- 2] 


[ATH 1 -4/5 -3/5 
sr secs] 
0:-4/5 -35 
2 15 2/5 
and A,=PTA,P, =| -5 -11/25 48/25 
0 -27/25 61/25 


A, is in Hessenberg form, so P P, and the transformation is completed. 


18.8 Transform to Hessenberg form the matrix 


123 -6 
|203 4 
^^5332 1 

26 4 21 


The first iteration (К = 1) of Steps 18.5 through 18.8 yields 


a veseman ЙЫ 3] 


for which we have, from Problem 15.13, 
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18.9 


= -3/7 6/7 
R, =| -3/7 6/7 2/7 


67 2/7 317 


у 257977617 
a B-loi-»] 6/7 2/7 
6/7 27 317 
boo 0 o 
ud д,=вїдр=|77 20499 -115149 27149 


0 -115/49  -64/49 142/49 
0 27/49 142/49 227149. 


The second iteration (k = 2) yields 


x,=[ 25] for which Xl; = VETT = 2.410756 
27149 

» _[ -115/49 [! е. КОБЫ] 
апд мех, іе, = | 271491 + 2410756 ol [658022 


for which |У, = 70:307696. Then, 


.[ 0.973528 -0.228567 
~ 1—0.228567 —0.973528 


R= 70228567 
01-0.228567 -0.973528 
t £3 0 0 
Сту p| -7 -2.326531 -2.410756 0 
and А: РРА) O -2410756 -2.285550 -1.273688 
O 0 -127%88 5612079 


Setting P=P,P,, we have P’A,P=A,, which is in Hessenberg form. Furthermore, since A, is 
symmetric, A, is tridiagonal. 


The (right) Kronecker product (or direct product) of an m x n matrix А = [a;] and a px q 
matrix B = [b,] is the mp х nq partitioned matrix 


Determine A@B when 


166 PATTERNED MATRICES [CHAP. 18 


18.10 Any n X m matrix X can be converted into an nm X 1 column vector x (denoted with a 
lowercase boldface letter) by taking the transpose of all the rows of X and placing them 
successively below one another into x. The matrix equation 


AXB-C (8.1) 
is then equivalent to the matrix-vector equation 
(A@B")x=c (18.2) 


where x and c are the vector representations of the matrices X and C, respectively. Equation 
(18.1) may be solved for the unknown matrix X in terms of A, B, and C by solving (18.2) for 
the vector x using the methods developed in Chapter 2. Equation (18.1) may possess exactly 
one solution, no solutions, or infinitely many solutions. 

Solve the matrix equation AXB = C for X when 


E 1 «| 
1 68 
a -6 75 
Let 
and 
Then (18.2) becomes 
2- 2 4 2 7 
її = - 0 1 
iS $ She 26 
6 3 в Mj 2 
-3 0 -4 O|."|-| 1 
8:9 4 dol v 68 
10 5 -2 [p | 35 
= 0 1 0 -6 
AS al und 75. 
Solving by Gaussian elimination, we find that x,, = 1, х, = 5, x;, = —1, and x,, = 2; hence, 
ы 1 35] 
x=[ | 2 


18.11 Solve the matrix equation АХВ = C for X when 
T 2] € -[1,2,7] 


A-[L2] B 


0 1 
efi za and а 4| 
ч. х2 $1 


Then (18.2) (Problem 18.10) becomes 


Xu 
о 10 2]. 1 
1 -1 2 -24 7 |=|2 
2 14 оу | 17 


Solving by Gaussian elimination, we find that this system has infinitely many solutions given by 
хи 7 3 2x,, and х,, = 1 —2x,, with x,, and x, arbitrary. Therefore, 


x-(?5 1-2] 


Xn Xn 


Let 
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Supplementary Problems 


18.12 Determine whether the following matrices are circulant, Toeplitz, band matrices, tridiagonal, and/or in 
Hessenberg form: 


“1 
(a) | 0 
0 
0° 
1 
(4) [; 
2 
= 
(8) о 
0 


2 
1 
3 


M 00 1010 1010 
0101 1010 

d 4 9| 0 рото (0 отот 

0 0-13 0101 0011 

3 120 123 

2 e |312 Фф [321 

1 0 1 3: 

2 0 0 

1 1 Q 

2-2 

0 1-1 


18.13 Find the eigenvalues of, and a canonical basis for, the matrix in Problem 18.12(5). 


18.14 Find the eigenvalues of, and a canonical basis for, the matrix in Problem 18.12(d). 


18.15 Determine the eigenvalues of the matrix in Problem 18.12(e). 


18.16 Construct an LU factorization for the matrix in Problem 18.12(e). 


18.17 Construct an LU factorization for the matrix in Problem 18.12( g). 


In Problems 18.18 to 18.20, transform A, into Hessenberg form. 


2 
18.18 А„=|1 
1 


x E 


0 0 311 1 
5 2| 1819 A,-|1 5 1| 18.20 A,-| 5 
113 1 


о-- - 
0 


18.21 Ап alternative procedure for transforming а matrix into Hessenberg form is Given's method, which 
utilizes rotation matrices (see Problems 15.25 through 15.27). The method is iterative, transforming to 
zero one element below the subdiagonal at a time. Zeros are introduced from left to right and from the 
last row up to the third row. For an n X n matrix A, with n 2 3, Given's method is as follows: 


STEP 1: 
STEP 2: 
STEP 3: 
STEP 4: 
STEP 5: 


STEP 6: 
STEP 7: 


Initialize counters k and i to k =n and i=1. 
Set j=i+1. 
Determine 6 so that tan @ = a,//a,;. 
Construct the rotation matrix R, (6). 
Calculate R/(6)AR, (6) and designate the product as the new matrix A. It will have a zero in 
the (k, i) position. 
If i k — 2, then increase i by 1 and return to Step 2; if i= k — 2, go to Step 7. 
If k = 3, stop; the algorithm is completed. If > 3, then decrease k by 1, set і = 1, and return 
to Step 2. 
Use this algorithm to reduce the matrix in Problem 18.7 to Hessenberg form. 


18.22 Use the algorithm given in Problem 18.21 to reduce the matrix in Problem 18.8 to Hessenberg form. 
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18.23 Construct СЮ when 


c-[i 3] ma H3 


18.24 Rework Problem 18.11 with C = [1, 2, 3]. 


18.25 Solve the matrix equation AXB =C for X when 


18.26 Solve the matrix equation AXB =C for X when 


10 1 1 1 
A=|2 1 1 В= [1,1,1] c= 1 1 
L3 =3 -2 
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Chapter 19 


Power Methods for Locating Real Eigenvalues 


NUMERICAL METHODS 


Algebraic procedures for determining eigenvalues and eigenvectors, as described in Chapter 7, 
are impractical for most matrices of large order. Instead, numerical methods that are efficient and 
stable when programmed on high-speed computers have been developed for this purpose. Such 
methods are iterative, and, in the ideal case, converge to the eigenvalues and eigenvectors of 
interest. Included with each method are termination criteria, generally a test to determine when a 
specified precision has been achieved (if the results are converging) and an upper bound on the 
number of iterations to be performed (in case convergence does not occur). 

This chapter describes algorithms for locating a single real eigenvalue and its associated 
eigenvector. The first method presented is the simplest; the last is the most powerful. Chapter 20 
describes a procedure for obtaining all eigenvalues of a matrix; it is usually packaged with the shifted 
inverse power method as an excellent general-purpose algorithm. 


THE POWER METHOD 


Applied to a matrix A, the power method consists in choosing a vector X and forming the 
sequence 
суХ, c, AX, c; A'X, c,A°X,... 
where со, С, C3, . .. are scaling constants selected to avoid computer overflow due to extremely large 
vector components. The sequence will generally converge to an eigenvector of A, and if the scaling 
constants are wisely chosen, the eigenvalue will be obvious too. This eigenvalue is usually the 


dominant eigenvalue of A, the one having greatest absolute value, provided such an eigenvalue is 
real. The usual implementation of the power method is as follows: 


STEP 19.1: Initialize X, so that its largest component in absolute value is 1, and initialize A, = 0 as 
the first approximation to the eigenvalue. Specify a desired precision PRE for the 
eigenvalue, and the maximum number of iterations to be performed; set an iteration 
counter k — 1. 

STEP 19.2: Calculate Y, = AX, ,. 

STEP 19.3: Determine the component of Y, that is largest in absolute value. Denote it as A,. 

STEP 19.4: Set X, = (1/A,)Y,. 

STEP 19.5: 1 |А, — A, ,| < PRE, stop; the eigenvalue and associated eigenvector are A, and X,. 
Otherwise, continue. 

STEP 19.6: Increase k by 1. If k is greater than the maximum number of iterations to be 
performed, stop. Otherwise, return to Step 19.2. 


(See Problems 19.1 through 19.4.) 

The power method will not converge if the dominant eigenvalue is complex. (See Problem 19.5.) 
Once convergence occurs, however, the procedure may be attempted again on A — AI to determine a 
second eigenvalue-eigenvector pair. (See Problem 19.3.) 

Convergence toward an eigenvalue may occur without an accompanying convergence of the 
associated eigenvector. A component that oscillates in sign with decreasing magnitude is converging 
to zero. 
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THE INVERSE POWER METHOD 


The inverse power method is the power method applied to A^, provided the matrix is 
nonsingular. The procedure will converge to the dominant eigenvalue of A’, the reciprocal of which 
is the eigenvalue of A having the smallest absolute value. The associated eigenvector is the same for 
both (Property 7.4). The steps are identical to those of the power method with the exception of the 
following: 


STEP 19.2: Calculate Y, = A 'X, , by solving the system AY, = X, , using LU decomposition. If 
this system does not have a unique solution, stop; zero is an eigenvalue of A. 


(See Problems 19.8 and 19.9.) 


THE SHIFTED INVERSE POWER METHOD 


The inverse power method may be used to find all real eigenvalues of a matrix if estimates of 
their locations are available. If и is an estimate of A, then A — ul will have an eigenvalue near zero, 
and its reciprocal will be the dominant eigenvalue of (A — uI) '. Therefore, if A and X are the 
eigenvalue and eigenvector obtained by applying the inverse power method to А — ul, then u+1/A 
and X are approximations to an eigenvalue and eigenvector of A. (See Problem 19.11.) 


GERSCHGORIN'S THEOREM 


Each row of a square matrix generates a Gerschgorin disk, which is bounded by a circle, whose 
center is the diagonal element in the row and whose radius is the sum of the absolute values of all 
other elements in that row. 


Example 19.1 The Gerschgorin disks for 


15 3+i4 0 
A=| -3 32 1-i3 
001 -025 2-i3 


are |: — 15] [3 +44] + |0] = 5 
|z - i32| s |-3| + [1 —43| = 6.162 
and |z - (2 - i3)| = [0.01] + |-0.25| = 0.26 


Property 19.1: (Gerschgorin's theorem) Every eigenvalue of a matrix (real or complex) must lie in 
one of its Gerschgorin disks. Furthermore, if the union of N of these disks is disjoint 
from all the rest, then there are exactly N eigenvalues in the union of those N disks. 


Gerschgorin's theorem is used to estimate the locations of the eigenvalues of a matrix. (See 
Problem 19.10.) Moreover, since the eigenvalues of a matrix are preserved under transposition, a 
second set of estimates may be developed by applying Gerschgorin's theorem to the matrix 
transpose. Still other estimates are provided by (12.3). 
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Solved Problems 


19.1 Use the power method to locate an eigenvalue and eigenvector for 


S cL 
A-|-1 -1 1 
Z o w5 


We choose X, = [1, 1, 1]”. Then we have: 
First iteration: 
Y, = AX, = (11, -1, 13]7 
LI 
x= i Y, = [0.846154, 0.076923, 1.000000]" 
i 
Second iteration: 
Y, = AX, = [11.307692, 0.230767, 10.846157)” 
А, = 11.307692 
X,= x Y, = [1.000000, 0.020408, 0.959184] 
2 
Third iteration: 
Y, = AX, = [11.693874, -0.061220, 11.816237)” 
Ay = 11.816327 
X,= x Y, = (0.989637, —0.005181, 1.000000)” 
" 


Continuing in this manner, we generate Table 19.1, where all entries are rounded to four decimal 
places. The algorithm is converging to the eigenvector [1, 0, 1]' with corresponding eigenvalue А = 12. 
Note how the second component of the eigenvector oscillates in sign as it converges to zero. 


Table 19.1 


Iteration Eigenvector components Eigenvalue 


1.0000 1.0000 1.0000 

0.8462 —0.0769 1.0000 13.0000 
1.0000 0.0204 0.9592 11.3077 
0.9896 —0.0052 1.0000 11.8163 
1.0000 0.0013 0.9974 11.9534 
0.9993 —0.0003 1.0000 11.9883 
1.0000 0.0001 0.9998 11.9971 
1.0000 —0.0000 1.0000 11.9993 
1.0000 0.0000 1.0000 11.9998 


19.2 Use the power method to determine an eigenvalue and eigenvector for 


5 2: 2 
A=|3 6 3 
669 


We initialize by choosing X, = [1, 1, 1]". Then we have: 
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First iteration: 
Y, = AX, =[9, 12, 21]” 


A, =21 
x= x Y, = [0.428571, 0.571429, 1.000000]7 
1 
Second iteration: 
Y, = AX, = [5.285714, 7.714286, 15.000000]” 
A, =15 
X, = 1 Y, = 0.352381, 0.514286, 1.000000]” 
Third iteration: 


Y, = AX, = (4.790476, 7.142857, 14.200000]” 
Ay = 14.2 


x,= x Y, = [0.337357, 0.503018, 1.000000]” 


Continuing in this manner, we generate Table 19.2, where all entries are rounded to four decima! places. 
The algorithm is converging to the eigenvector [1/3, 1/2, 1]’ with corresponding eigenvalue А = 14. 


Table 19.2 
[7INCTTTT NETTE 
1.0000 1.0000 1.0000 


0.4286 0.5714 1.0000 21.0000 


Use the power method to determine a second eigenvalue for the matrix given in Problem 19.1. 


Instead of the matrix A given in that problem, we consider the matrix A — 121, which has the same 
eigenvectors as A but whose eigenvalues are those of A reduced by 12. One eigenvalue of A — 121 is, 
therefore, zero, corresponding to the eigenvalue found in Problem 19.1. Since zero most likely is not the 
dominant eigenvalue of A — 121, we can use the power method on this new matrix to locate another 
eigenvalue and eigenvector. We set 


=<) -f 7 
B-A-MI-|-1 -13 1 
7 T =7 


and apply the power method to В. The results are summarized in Table 19.3. The algorithm is 
converging to the eigenvector [1, 1, —1]" with associated eigenvalue А = —15. The corresponding 
eigenvalue for the matrix A is A= —15 + 12 = —3 with the same associated eigenvector. 

Having two eigenvalues of a 3 х 3 matrix, we can produce the third easily. The trace of A equals the 
sum of the eigenvalues, so 5 + (-1) +5 = 12 +(—3) + A. The last eigenvalue of A is thus А = 0. 
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Table 19.3 


Iteration Eigenvector components Eigenvalue 


1.0000 1.0000 1.0000 

0.0769 1.0000 — —0.0769 | —13.0000 
0.1579 1.0000 -—0.1579 | —13.1538 
0.2411 1.0000 -0.2411 — 13.3158 
0.3245 1.0000 0.3245 — 13.4822 
0.4061 1.0001  —0.4061 — 13.6491 


0.7348 1.0000 — —0.7348 —14.3651 
0.9662 1.0000 —0.9662 —14.9160 
0.9963 1.0000 —0.9963 —14.9907 
0.9996 1.0000 — —0.9996 —14.9990 
0.9999 1.0000 — —0.9999 —14.9997 


19.4 Derive the power method. 


19.5 


Assume that the matrix A has order n x n and possesses n real eigenvalues A,, À;, . . . , А, such that 
lanl 
Furthermore, assume that the eigenvectors V,, V,,..., V, corresponding to each of these eigenvalues 


form a linearly independent set. Then for any n-dimensional vector X there exist constants 
d,,d,,...,d,, not all zero, such that 


X=d,V, + dV, * dV, 


IAL > [As] 


Multiplying on the left by A repeatedly, we get 
АХ = d AFV, + d,ASV, +++ + d AAV, 


A'X PE AAS 
or АХ ам) а (2) v, 

Ae td) Ma a, 
Since A, is the dominant eigenvalue, the sequence converges to d,V,. Therefore, the power method will 
converge to the eigenvector associated with the dominant eigenvalue provided (1) the dominant 
eigenvalue is real and of multiplicity one, and (2) the initial guess X is not a linear combination of the 
remaining n — 1 eigenvectors (d, #0). The rate of convergence is a function of the ratio |A,/A,|. 


Apply the power method to 


and explain the result. 


Applying the power method to this matrix, we generate Table 19.4, from which we conclude that 
the algorithm is not converging. The reason is that the eigenvalues of A are 1, —2 + i6, and —2—16, 
with the dominant ones being complex. 
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Table 19.4 


Iteration Eigenvector components 


1.0000 1.0000 
—0.4000 0.1000 
0.5455 1.0000 
—0.8076 —0.4290 


0.7229 1.0000 
—0.6691  —0.1957 
0.2492 1.0000 
1.0000 0.8220 


19.6 A modification of the power method particularly suited to real symmetric matrices is 


initialized with a unit vector (in the Euclidean norm) having all its components equal. 

At each iteration Y, is determined as before, but the eigenvalue is approximated as 
А, = X, ,*Y,, an approximation to the Rayleigh quotient. Then X, =Y,/|lY,||,, unless 
|У, |, = 0, in which case zero is an eigenvalue and the algorithm is terminated. Use this 
modified power method to determine an eigenvalue and eigenvector for 


007 8 7 
75 6 5 
A=] 3 6 10 9 
75 9 10 


We initialize with X, = [0.5, 0.5, 0.5, 0.5]", which is the vector [1, 1, 1, 1]" normalized. Then, with 
all calculations rounded to four decimal places, we have: 


First iteration: 
Y, = AX, = [16, 11.5, 16.5, 15.5] 
м, = X, Y, 729.75 
IY, ll: = 30.0125 
X, ы = (0.5331, 0.3832, 0.5498, 0.5165]7 
Second iteration: 
= (16.0267, 11.5285, 16.7097, 15.7601]” 
Y, = 30.2873 
(0.5291, 0.3806, 0.5517, 0.5203] 
Third iteration: 


Y, = АХ, = (16.0118, 11.5191, 16.7170, 15.7759]" 
As =X," Y, = 30.2887 
{ү Il, = 30.2887 


Y 
X,- Y = [0.5287, 0.3803, 0.5519, 0.5209]" 
allz 


At this point, the eigenvector is already accurate to four decimal places. The standard power 
method would take six iterations to achieve similar accuracy. 
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19.7 


19.8 


Use the modified power method described in Problem 19.6 to determine a second eigenvalue 
and associated eigenvector for the matrix in Problem 19.6. 


Having determined that 30.2887 is an eigenvalue of A, we can apply the modified power method to 


-20.2887 7 8 7 

7 25.2887 6 5 

а ag 6 — -2287 9 
7 5 9  -202887 


We initialize with X, = [0.5, 0.5, 0.5, 0.5]". Then with all calculations rounded to four decimal places, we 
have: 


First iteration: 
Second iteration: 
Y, = BX, = [-7.3891, 27.0345, - 9.8380, ~ 1.8130] 
à; =X, Ya = -29.7275 
lIY.ll, = 29.7579 
X, = 72- = [-0.2483, 0.9085, 0.3306, -0.0609]" 
ША 


Continuing in this manner, we generate Table 19.5. Four-place precision is attained by iteration 
number 65, although it takes quite a few additional iterations before confidence in the result is 
established. The algorithm is converging to —30.2785, so a second eigenvalue of the original matrix is 
— 30.2785 + 30.2887 = 0.0102. 

At this point, however, there is no convergence to an eigenvector, because the sign of each 
component is changing at each iteration and none of the components has stabilized. This suggests a third 
eigenvalue very close to the second one. An alternative approach to finding the desired eigenvector is 
given in Problem 19.9. 


Table 19.5 


Iteration Eigenvector components Eigenvalue 


0.5000 0.5000 0.5000 0.5000 

0.2141 -0.9117 0.3391 0.0890 —0.5387 
—0.2483 0.9085 —0.3306 -0.0609 | -29.7275 

0.2788  —0.9040 0.3222 0.0357 29.8433 


0.4981  —0.8294 0.2170 -0.1299 —30.2784 
—0.4982 0.8294 -0.2168 0.1298 —30.2784 
0.4983 —0.8294 0.2165  —-0.1296 —30.2785 
—0.4984 0.8295 -0.2163 0.1294 ~ 30.2785 


Use the inverse power method to determine a second eigenvalue and second eigenvector for 
the matrix in Problem 19.2. 


For this matrix, LU decomposition yields 
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[ 0 «| I 0.4 EJ 
L-[3 48 0 and U-|0 1 0375 
6 3.6 525 00 1 
With X, =[1, 1, 1]7, the algorithm yields the following: 
First iteration: Solve LZ, = X, to obtain 
Z, = [0.200000, 0.083333, —0.095238]" 
Solve UY, = Z, to obtain 
Y, = [0.190476, 0.119048, —0.095238]" 
A, = 0.190476 
x= x Y, = [1.000000, 0.652500, —0.500000]7 
Second iteration: Solve LZ, =X, to obtain 
Z, = (0.200000, 0.005208, -0.327381]” 
Solve UY, = Z, to obtain 
Y, = (0.279762, 0.127976, –0.327381]' 
A, = -0.327381 
X,= i Y, = [-0.854545, 0.390909, 1.000000]” 
Third iteration: Solve LZ, = X, to obtain 
Z, = [-0.170909, 0.025379, 0.368398]7 
Solve UY, = Z, to obtain 
Y, = [-0.273160, —0.112771, 0.368398)” 
А, = 0.368398. 


X, x Y, = [-0.741481, -0.3061 10, 1.000000)” 


Continuing in this manner, we generate Table 19.6, where all entries are rounded to four decimal 
places. The algorithm is converging to an eigenvalue of 1/3 for A`', or its reciprocal 3 for A. The 
associated eigenvector is the same for both A“! and A; it is converging toward [-0.7143, 0.2857, 1]’. 

Having produced two eigenvalues for a 3X 3 matrix, we can obtain the third one easily. Using 
Property 7.1 and the results of Problem 19.2, we have 5+6+9 = 14 +3 + A, so the last eigenvalue is 
also 3. 


Table 19.6 


Iteration Eigenvector components Eigenvalue 


1.0000 1.0000 1.0000 

1.0000 0.6250  -0.5000 0.1905 
—0.8545 ~0.3909 1.0000 —-0.3274 
—0.7415  —0.3061 1.0000 0.3684 
—0.7200 -0.2900 1.0000 0.3401 
—0.7155  —0.2866 1.0000 0.3348 
—0.7145  —0.2859 1.0000 0.3336 
-0.7143  —0.2858 1.0000 0.3334 
-0.7143  -0.2857 1.0000 0.3333 
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19.9 Use the inverse power method to obtain an eigenvalue and eigenvector for the matrix in 
Problem 19.6. 


For this matrix LU decomposition yields 


10 0 0 0 1 0.7 08 0.7 
7010 0 01 4 1 
8042 0| 20 U=lo 1 is 
7 01 3 05 3 Lo x 0 <a 


With X, = [1, 1, 1, 1], the algorithm yields the following: 
First iteration: Solve LZ, =X, to obtain 
Z, = [0.100000, 3.000000, –0.500000, 3.000000]” 
Solve UY, = Z, to obtain 
Y, = [—12.000000, 20.000000, — 5.000000, 3.000000]” 
А, = 20.000000 
X,- xi = [—0.600000, 1.000000, 0.250000, 0.150000]" 


Second iteration: Solve LZ, = X, to obtain 
Z, = [-0.060000, 14.200000, –2.725000, 14.650000]” 
Solve UY, = Z, to obtain 
Y, = [—59.400000, 98.350000, —24.700000, 14.650000]" 
А, = 98.350000 


X,- l Y, = [-0.603965, 1.000000, 0.251144, 0.148958]" 
2 


Two more iterations yield 


X, = [-0.603972, 1.000000, —0.251135, 0.148954)” 
and X, = [—0.603972, 1.000000, — 0.251135, 0.148953]” 


with A, = 98.521606 and A, = 98.521698. The fifth iteration is identical to the fourth, so X, approximates 
an eigenvector of A corresponding to the eigenvalue 1/98.521698 = 0.010150. 


19.10 Use Gerschgorin’s theorem to estimate the eigenvalues of 


29 -1 4 
A-|-1 -18 2 
4 21 


The Gerschgorin disks are 
|: - 29] ж |-1| + |4 75 
|: +18] s |-1] [2] 23 
|: - 1154 + 2176 
Since A is a real symmetric matrix, its eigenvalues must be real (Property 13.3), and the Gerschgorin 


disks reduce to the intervals 245 z = 34, —21 < z = - 15, and —5 = z < 7. Furthermore, these intervals 
(disks) are disjoint, so there must be one eigenvalue in each. 
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Find the eigenvalues and a corresponding set of eigenvectors for the matrix in Problem 19.10. 


From Problem 19.10, we know that one real eigenvalue is located in the interval 24 = z $34. We 
take и = 28 as an estimate of this eigenvalue and apply the inverse power method to A — 281. A better 
estimate for the eigenvalue might be the center of the interval, u —29, but an LU decomposition for 
A — 291 is not possible because that matrix has a zero in the (1,1) position. For A — 281, we have 


1) 0 0 1 i 4 
L=|-1 -47 0 and U-|0 1 -0.127660 
4 6  —42.234043 0 0 1 


Applying the inverse power method with these matrices, we obtain, after five iterations, Х, = 
[1.0, —0.015180,0.138939]" with A, = 0.636563. The corresponding eigenvalue for A is A= 28+ 
1/0.636563 — 29.5709. 

From Problem 19.10 we know that a second real eigenvalue lies between —15 and —21. We estimate 
this eigenvalue as и = —19. The LU decomposition for A + 191 has 


1 -0.020833 0.083333 
0 1 2.127660 


48 0 0 
L2|-1 0.979167 0 and U= 
0 0 1 


4 2.083333 15.234043 


With these matrices the inverse power method yields, after five iterations, X,= 
[0.030495, 1, —0.110227]" with A, = 1.335023. The corresponding eigenvalue for A is -19+ 
1/1.335023 = — 18.2509. 

The last real eigenvalue is between —5 and 7. We estimate it as и = 0, and apply the inverse power 
method directly to А. After five iterations we find X, = [—0.137203, 0.114411, 1]” with A, = 1.470566. 
The corresponding eigenvalue for A is 1/1.470566 — 0.6800. As a check we note that the sum of the three 
eigenvalues is 29.5709 + (— 18.2509) + 0.6800 = 12, which is the trace of A. 


Prove that each eigenvalue of a square matrix A lies in at least one Gerschgorin disk 
generated by A. 


Let A be an eigenvalue of A = [a, ] corresponding to the eigenvector X = [x,, x3, . . . x, ]". Denote 
the largest component of X in absolute value as x,,. Then we have 


AX = AX a) 
and xul = тах({х,|, (1...6) (2) 
Equating the Mth components of both sides of (1) gives 


У аых,= Ах 
a 


so (А-а„„)х„= È aux, 
Pr 


It follows from this last equation and (2) that 


fixis E Па) X lau 
A ‘a 


là- амм = | Zar, 
— 


Thus А is in the Mth Gerschgorin disk, so it is certainly in one of them. 
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19.13 


19.14 


19.15 


19.16 


19.17 


19.18 


19.19 


19.20 


1921 


19.22 


19.23 


19.24 


Supplementary Problems 


Apply five iterations of the power method to 


Use the power method to locate a second eigenvector and eigenvalue for the matrix in Problem 19.2, 
Observe that convergence occurs even though that eigenvalue has multiplicity two. 


Apply the power method to the matrix in Problem 19.11 and stop after four iterations. 


Apply the power method to 


and show that convergence to the eigenvalue is rapid even though there is no convergence to the second 
component of the eigenvector. Deduce the value to which this second component is converging. 


Determine why the power method did not converge to the dominant eigenvalue in Problem 19.16. 


Apply the power method to 


Apply the power method to the matrix in Problem 19.6. 


Apply the modified power method to 


Apply three iterations of the modified power method to find a second eigenvector and eigenvalue for the 


matrix in Problem 19.20. 
405 
A=/1 4 2 
304 


Apply the inverse power method to the matrix in Problem 19.1. 


Apply the inverse power method to 


Apply three iterations of the inverse power method to 


-1 
a} 25 
Ac 3 
3 


19.25 


19.26 


19.27 
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The matrix in Problem 19.24 is known to have an eigenvalue near 9. Use the shifted inverse power 
method to find it. 


The matrix in Problem 19.18 is known to have an eigenvalue near 2.5. Use the shifted inverse power 
method to find it. 


A modification of the shifted inverse power method uses the Rayleigh quotient as an estimate for the 
eigenvalue and then shifts by that amount. At the kth iteration, the shift is А, = XTAX,/X; X,. Thus, the 
shift is different for each iteration. Termination of the algorithm occurs when two successive A iterates 
are within the prescribed tolerance of each other. Use this variable shift method on the matrix in 
Problem 19.20. 


Chapter 20 


The QR Algorithm 


THE MODIFIED GRAM-SCHMIDT PROCESS 

The Gram-Schmidt orthogonalization process (as presented in Chapter 11) may yield grossly 
inaccurate results due to roundoff error under finite-digit arithmetic (see Problems 20.10 and 20.11). 
A modification of that algorithm exists which is more stable and which generates the same vectors in 
the absence of rounding (see Problem 20.12). This modification also transforms a set of linearly 
independent vectors (X,, X; » X, ) into a set of orthonormal vectors {Q,,Q,,...,Q,} such that 
each vector ©, (k = 1,2, . . . , n) is a linear combination of X, through X,_,. The modified algorithm 
is iterative, with the kth iteration given by the following steps: 


STEP 20.1: Set r,, = ||X, ||, and О, = (1/r,,)X,. 

STEP202: Forj=k+1,k+2,...,n, set г, = (X. Q) 

STEP 20.3: For j=k+1,k+2,...,n, replace X, by X, – r,Q,. 
(See Problems 20.1 and 20.3.) 


QR DECOMPOSITION 


Every m Xn matrix A (mz п) can be factored into the product of a matrix Q, having 
orthonormal vectors for its columns, and an upper (right) triangular matrix R. The product 


A=QR (20.1) 


is the QR decomposition of A. If A is square, then Q is unitary. 

The QR decomposition follows immediately from the modified Gram-Schmidt process applied to 
the columns of A, provided those columns are linearly independent. If they are, then the columns of 
Q and the elements r; (i<j) of R are the quantities generated by the modified Gram-Schmidt 
process. (See Problems 20.2 and 20.4.) 

If the columns of A are not linearly independent, then one or more of the r,, values determined 
in Step 20.1 will be zero. That step must be modified to: 

STEP 20.1: Calculate r,, = ||X, ||,. If r,, #0, then Q, =X,/r,,. If r}, = 0, then choose Q, to be 
any normalized vector which is orthogonal to Q,, Q,, ..., Q,. ,. 

In practice, r,, is rarely zero. Even if the columns of A are linearly dependent, roundoff will 
produce an r,, value close to but not equal to zero, permitting Q, to be calculated in the usual 
manner. The result is however, an incorrect vector. (See Problem 20.13.) Thus, whenever an r,, 
value is sufficiently small, Q, must be checked to guarantee it is orthogonal to the previously 
calculated Q vectors; if it is not, the modification given as Step 20.1' must be implemented. 


THE QR ALGORITHM 


The QR algorithm is a procedure for determining all eigenvalues of a real matrix Ap. The 
algorithm sequentially constructs matrices A, (k = 1,2, 3, . . .) by forming QR decompositions 


Ari = Q,-,R,., (20.2) 
for A,_,, and then reversing the order of the products to define 
A, 7 R,-,Q,., (20.3) 
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Each A, is similar to its predecessor and has the same eigenvalues (see Problem 20.9). In general, 
the sequence {A,} converges to a partitioned matrix having either of two forms: 


(20.4) 


and (20.5) 


If form (20.4) occurs, then the element a is an eigenvalue, and the remaining eigenvalues are 
obtained by applying the QR algorithm anew to the matrix E. If form (20.5) arises, then two 
eigenvalues can be determined from the characteristic equation of the 2 x 2 submatrix in the lower 
right partition, and the remaining eigenvalues are obtained by applying the QR algorithm to the 
matrix С. If E or G is already a 2 X 2 matrix, its eigenvalues are determined from its characteristic 
equation. 


ACCELERATING CONVERGENCE 


Convergence of the QR algorithm is markedly accelerated by a shift at each iteration: If the 
matrices have order n X n, then the element in the (n, п) position of A, , is denoted as s, ,, and a 
QR decomposition is constructed for the shifted matrix A, ., — s, _,1. Equation (20.2) is modified to 


A,4 7 Seal = Q, AR, (20.6) 
and (20.3) is replaced with 
A, = R,4Q,., 5,.,l (20.7) 


Equations (20.6) and (20.7) constitute the shifted QR algorithm. (See Problems 20.5 through 20.8.) 

For matrices of large order, significant computation time is also saved by first reducing the given 
matrix to Hessenberg form and then applying the shifted QR algorithm to it. 

At each stage the QR (or shifted QR) algorithm is halted once the zeros in form (20.4) or (20.5) 
are obtained to whatever degree of precision is specified by the user. If the eigenvalues are real, only 
estimates are needed for them. These estimates are then incorporated into the shifted inverse power 
method (Chapter 19) to quickly obtain better values for the eigenvalue and corresponding eigen- 
vector. 
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Solved Problems 


20.1 Use the modified Gram-Schmidt process to construct an orthogonal set of vectors from the 
linearly independent set (X,, X,,X,} when 


-4 2 2 
X-|3| X-|-3| Х,=|3 
6 6 0 
First iteration: 


ти = |IX,ll, = VOI = 7.810250 
1 [ 4 3 6 n E 
mee al б о. | =. .384111, 0.768221 
Qez х, Voi’ Vet Ver) 7109512148, 0.384111, 0.768221] 
ra = (Xar) = уу = 2.432701 


nis = (X,Q,) = E 70.128037 
X, —X, - 7,.Q, = [198/61, —240/61, 252/61] 
X, —X, - 70, = [126/61, 180/61, —6/61]7 


Second iteration (using vectors from the first iteration): 


У160,308 
7,7 1х, = V(198/61Y + (240761): + (252/61). = ves = 6.563686 


ely [198 — 20 252 je 
@= Xx [o Vico 308" 171603081 = (0.494524, -0.599423, 0.629305]" 


324 

Tay = 065.0) = - veg = 70809222 
126/61 0.494524] [2.465753 
X, — X, - 74,Q, = | 180/61 | - (-0.809222)} —0.599423 | = | 2.465753 
-6/61 0.620395 | 0.410959 


Third iteration (using the vector from the second iteration): 
гуу = V (2.465753)! + (2.465753)? + (0.410959)? = 3.511234 


Q, =} X, = [0.702247, 0.702247, 0.117041]7 
7 


An orthonormal set is {Q,, Q3, Q;). 


20.2 Construct a QR decomposition for the matrix 


From Problem 20.1, we have 
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Q=| 0.384111 -0.599423 0.702247 0 6.563686 —0.809222 


—0.512148 0.494524 Ed [е 2.432701 sd 
and R= 
0.768221 0.629395 0.117041 0 0 3.511234 


A direct calculation shows that А = QR. 


20.3 Use the modified Gram-Schmidt process to construct an orthogonal set of vectors from the 
linearly independent set {X,, X3, X3, X4} when 


>< 
н 


X,= 


1 
0 
| = 
1 


‚ее 
cocco 


First iteration: 
ra 7 lIXill, = V3 


ru 7 (0) = 8 


X, =X, - 74Q, = [1, 72/3, 1/3, 1/3]" 
X, —X, - rQ, = [1,1/3, -2/3,1/3]" 
X, X, - uQ, = [1,1/3,1/3, -2/3] 


Second iteration (using vectors from the first iteration): 


ra = ЇХ,|„ = VOY + (72/3) + Q/3Y + (1/3). = Ув 


ra= (Х,.0,) = турк 


r= (Хо) = a 


1 З/У 3/5 
X-X,-rnQ9,-| 13|-.2.|-72v15|.| 3/5 
WT UY -2/3| vis| vvis -4/5 
1/3 1У13, 1/5. 
1 ЗМУ 3/5 
y3| 2 |-2wis$|.| 3/5 
1/3 1 vvis| | vs 
-2/3. 1/Vis. —4/5. 
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Third iteration (using vectors from the second iteration): 


т = Xll, = У(3/5)* + (3/5Y + (—4/5)° + (1/57 = MB 
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S. AJ 
5” V35 
335 3/7 
22 | эм3|_| 3/7 
V35 | -4/V35| | 3/7 
1/38. -6/7 
Fourth iteration (using the vector from the third iteration): 
ra = 1х1, = V/F + 77 + (3/7) + (7677) = ve 
g,-1x. -[5 з. з. Xl -[4 cadi E 
“оты * LV63 Vel VOR’ V63 VT VT VI’ VI 
An orthonormal set is {Q,, Q2, Qs, Q4}. (Compare with Problem 11.7.) 
20.4 Construct a QR decomposition for the matrix 
0 11 1 
Z3. 0, 1 1 
А=|11от 
1110 
Using the results of Problem 20.3, we have 
0 3/V15 3/35 1/77 УЗ 2N3 2N3 23 
g=| ИМЗ 2/15 3У35 17] gga] 0 VIS 2715 2/VIS 
ПУЗ 1V5 -4/V35 1/77 0 0 У35/5 2IV35 
ПУЗ qvis 135 -2v7 0 0 0 vein 
A direct calculation shows that А = QR. 
20.5 Apply the shifted QR algorithm to 
[3 1 
A s 
Using (20.6) and (20.7), we calculate the following: 
First iteration: 
[221 _ [ -0.894427 0.447214][2.236068 -0.894427 
Asie [71 o= em. -[ 70 ed | 0 0.447124 
а= —0.894427 || —0.894427 nanu i 0]- [26 0.2 
"CL g 0.447214 ]L 0.447214 0.894427 1] (02 54 
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]-e R= [ 1545. 0 DAT [c mis Ec 
at 0.071247 0.997459 0 0.014249 


[ps —0.199492 [795959 КОА] +з]! 0 
0 0.014249 0.071247 0.997459 "40 1 


A, =R,Q, + 5.41= 


R 585787 0.001015 
0.001015 5.414213 


Third iteration: 


A, - 5.414213 = [ —2.828426 0.001015 —1.000000 жес [sar Шр 


0:001015 0 ]-ar, [ИУ ОК a 0.000000 


_ _ [2.828427 -0.001015][ -1.000000 0.000359 
Ay B0, 5:41214E [ 0 ООЙ. 0.000359 1.000000 
-[ жн -0:000000] 

0.000000 5.414213 


] + saafi °] 


At this point we have generated form (20.4). It follows that one eigenvalue is 5.414213 and the 
second is 2.585786. (Observe the roundoff error in Q,, which results in columns that are only 
approximately unit vectors.) 


20.6 Apply the shifted QR algorithm to 


522 
A,7|3 6 3 
6 69 
Using (20.6) and (20.7), we obtain the following: 


First iteration: 
-4 22 

A,-91=| 3 -3 3|-QR, 
6 60 


0.384111 —0.599423 0.702247 0 6.563686 —0.809222 
0.768221 0.629395 0.117041 0 0 3.511234 


озан 0.494524 пло o 2.432701 Ld 


as already shown in Problem 20.2. Then we have 


6.032787 2.484726 7.208066 
A, =R,Q, + 91=| 1.899520 4.556254 4.514615 
2.697405 2.209952 9.410959 


Second iteration: 


—3.378172 2.484726 7.208066 
А, —9.4109591 =| 1.899520 -4.854705 4.514615 | = QR, 
2.697405 2.209952 0 


[саш 0.134590 0.685600 [v —2.468146 rmm 


0.402280 -0.722952 0.561704 0 5.341738  -2.293716 
0.571256 0.677662 0.463077 0 0 7.477730. 


3.131498 0.155992 0.303949 
A, = Е,0, +9.4109591 = 0.838575 3.994775 1.983808 
4.271696 5.067376 12.873727 


Continuing in this manner, we generate 
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3.174780 0.197407 4.602285 
А, = | 0.127357 3.143844 3.353551 
0.405644 0.458158 13.681376 
2.995229 -0.005512 -4.152641 
A, =} -0.003362 2.996115 -2.926538 
0.012647 0.014613 14.008656. 
—0.000004  —4.166180 
А, = E: ООШ 2.999997 —2.939863 
0. мге 0.000011 14.000007 
— 0.000000 -4.166189 
ipee 3.000000 —2.939868 
0.000000 0.000000 14.000000. 


Convergence is established to the number of decimal places shown. One eigenvalue, 14, appears in the 
(3,3) position; it is obvious that the other two eigenvalues are 3 and 3. (Compare with Problem 19.2.) 


20.7 Apply the shifted QR algorithm to 


i 32 =f 
-h 123 
АБ а: ў 
2-21 2 


After fifteen iterations of the algorithm, we obtain 


— 1.942221 0.528619 1.156169 0.023944 

ГЕ С 1.278276 —1.310739 0.878963 1.339487 
ш 0.000000 0.000000 4.639537 —2.951749 
0.000000 —0.000000 0.630294 3.613422 


which has form (20.5). The characteristic equation of the lower right 2 x 2 matrix 


—2.951749 
3.613422 


4.639537 
0.630294 


is А? — 8.252959A + 18.625075 = 0, which has as its roots the eigenvalue estimates 4.126480 + 1.263820. 
The upper left 2 x 2 submatrix has the characteristic equation A^ + 3.252960A + 3.221466 = 0, which has 
as its roots the eigenvalue estimates — 1.626480 + 0.758965. 


20.8 Apply the shifted QR algorithm to 
10; 1.:8- i 
al 738 © 5 
^^|s 6 10 9 
75 9 10 
After four iterations of the algorithm, we obtain 
30.288604 0.049230 0.006981 –0.000000 
Rs 0.049230 0.021553 0.096466 0.000000 
* 0.006981 0.096466 0.831786 —0.000000 
0.000000 —0.000000 0.000000 3.858057. 
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This matrix has form (20.4), so one eigenvalue estimate is 3.858057. To determine the others, we apply 
the shifted QR algorithm anew to 


30.288604 0.049230 0.006981 
Е, = | 0.049230 0.021553 0.096466 
0.006981 0.096466 0.831786. 


After two iterations, we obtain 


0.000038 0.010150 0.000000 
0.000000 0.000000 0.843107 


so a second eigenvalue estimate is 0.843107. The characteristic equation of the upper left 2 х 2 submatrix 
in E, is A? — 30.298836A + 0.307430 = 0, which can be solved explicitly to obtain 30.288686 and 0.010150 
as the two remaining eigenvalues. (Compare these values with those obtained in Problems 19.6 and 
19.7.) 


К 0.000038 coma 


Show that the shifted QR algorithm is a series of similarity transformations that leave the 
eigenvalues invariant. 


Since the Q matrix in any QR decomposition is unitary, it has an inverse. Therefore, (20.6) may be 
rewritten as 


Ry =Q (4, Z 5,.,D) 
Substituting this equation into (20.7), we obtain 
A, = QE (A,.,7 5,,DQ, +з,_,1 = QA, .,0,., 75, 40, IQ, + 54-1 
70,54,.,0,., 
Therefore, A, is similar to A, _,, and the invariance of their eigenvalues follows from Property 10.1. 


Working to four significant figures, show that the unmodified Gram-Schmidt orthogonalization 
process does not generate an orthogonal set of vectors when applied to 


1 
1 

X=] 1.01 X= 
1 


1 1 
1.01 1 
Maly 1 
1 1.01 
Using the algorithm given in Chapter 11 and rounding all stored numerical values to four significant 
digits, we obtain 


VG, X,) = 2.005 
so ©, = chs X, = [0.4988, 0.5037, 0.4988, 0.4988)” 
Then Y, =X,- (X2,Q,)Q, = X, – 2.0050, 


= [-0.9400 x 107*, –0.9919 x 107°, 0.9906 x 107°, —0.9400 x 107*]^ 


and V(Y,. ¥,) = 0.1402 x 107" 
М 1 

Q= баб x10 

Lastly, Y, = X, - (Х,,0,)0, - (X;, Q,)Q, = X, - 2.0050, – (-0.1438 x 107')0, 


= [70.1904 x 1077, —0.2009 x 107, 0.1007 x 107, 0.9810 x40 7]" 


ү, = [70.6705 x 107° —0.7075, 0.7066, —0.6705 x 1077]" 
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20.12 


and V(Y,. Ү,) = 0.2452 x 107! 
1 


= 0282x107 Y, = [70.7765 x 107°, 0.8193, 0.4107, 0.4001]” 


so о, 


For these vectors, (Q,, ©.) = 0.8672, which is not near zero as it should be. Similar results are obtained 
wherever the components 1.01 are of the form 1 + 10^* and all numerical values are rounded to 2k 
significant digits. 


Redo Problem 20.10 using the modified Gram-Schmidt process and show that the results are 
better. 


First iteration: 
п = [IX, |], = 2.005 
„Ъз е А 
Q, = z ppg X. = (0.4988, 0.5037, 0.4988, 0.4988] 
Tha = (X3, Q1) = 2.005 
т = (Х,,О,) = 2.005 
X, X, - 2.0050, = [-0.9400 x 107*, —0.9919 x 107, 0.9906 x 107°, -0.9400 x 1074] 
X, «- X, — 2.0050, = [—0.9400 x 107*, —0.9919 x 107°, —0.9400 x 1074, 0.9906 x 1072]7 


Second iteration: 
т» = 1,1, = 0.1402 x 107" 


"—— ыер [= "ul е -nr 
Q: 01402107 * [70.6705 x 107°, —0.7075, 0.7066, -0.6705 x 107] 


Ta = (X4, Q;) = 0.6885 x 10°? 
X, «- X, — 0.6885 x 10720, = [0.4784 x 107*, —0.5048 x 107°, —0.4959 x 1077, 0.9952 x 107?]" 
Third iteration: 
ry = ||X,l|; = 0.1221 x 107! 


1 
ozi xio % 


= [0.3918 x 107°, —0.4134, —0.4061, 0.8151] 


о, 


For these vectors, (Q,, Q,) = 0.00003872, which is much better than the result obtained in Problem 
20.10. All other inner products formed from the vectors obtained here are at least accurate as those 
formed from vectors found in Problem 20.10. 


Show that the modified Gram-Schmidt process yields the same vectors as the unmodified 
process in the absence of rounding 


The proof is by induction. Q, is the same for both methods because it is computed in exactly the 
same way in both algorithms. Assume that the two methods provide identical results for 
0,.0,,...,0,. We need only show that they do so for Q,,, as well. 

It follows from the induction hypothesis that Q,,Q,,...,Q, are mutually orthogonal because they 
are the vectors obtained from the unmodified Gram-Schmidt process. Let ХХ (i > j) denote the value of 
X, after it has been altered by j iterations of the modified process. Then, for the modified process, we 
have 
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XP = Xea 7 (Xe QQ а) 
X22, = Xe - (322, 0,)0, (2) 
X2, 7 X) - (XL, QQ. (3) 


X?) хело 05.90, 


Substituting (1) into (2) and noting that Q, and Q, are orthogonal, we obtain 


XE = Kee (Х,.,.0,)0, – (X, 7 05, 0,)0,), 0,)О, 
2X4 (Х,.1.0,)0, - (Х,.. 0,)0, + (Х,,..0,)(0,.0,)0, 
=X yor 7 (X,..0)0, - (Ky 1, QW 


Substituting this result into (3) and noting that Q, is orthogonal to both Q, and Q,, we obtain 


хел 7X, 7 (Xuri QQ = (Х,.,,0,)9, - (Kas 7 (5.5 QQ. 7 (Х,.1,0,)0,), О,)О, 
7X, 7 (Х,,1.0,)0, - (X,.,,0))0; - (Х,,,.0,)0, + (X,,,,0,)(0,.0,)0; 
+ (X,.,,0,)(0,,0.)0, 
= Kao T (Kory Qi DQ, - (X,.,,.0))0, - (X,.,,Q.)Q, 


Continuing in this manner, we find that X7"), is identical to Y,,, in the unmodified Gram-Schmidt 


process, and, since Ф, , , is obtained in one method by normalizing X^!, and in the other by normalizing 
Y, „1, it follows that Q,,, is the same in both methods. 


101 
A=|0 1 1 
1 1 2 


working to six significant digits, and show how roundoff error can generate an incorrect Q 
matrix when the columns of A are linearly dependent. 


20.13 Construct a QR decomposition for 


Designate the columns of A as X,, X,, and X, from left to right. Then: 


First iteration: 

ra = Xil, = V2 = 0.141421 x 10' 

X, = (0.707109, 0, 0.707109’ 

Tz = (X4, Q,) = 0.707109 

7,57 (X,, Q,) = 0.212133 x 10" 

X, —X, - rQ, = [-0.500003, 1, 0.499997)” 

X, — X, - rQ, = [-0.500012, 1, 0.499988]” 
Second iteration: 


та = 1,1, = 0.122474 x 10! 
Q: =+ X, =[—0.408252, 0.816500, 0.408247)” 
22 


тэ = (X4, Q2) = 0.122475 x 10' 
X, — X, — „0, = [—0.536300 x 107°, -0.837500 x 107°, -0.125133 x 107*]7 


CHAP. 20] THE QR ALGORITHM 191 


Third iteration: 
753 = ЇХ.||„ = 0.159839 x 107* 


о, -3 X, = [-0.335525, —0.523965, —0.782869]" 
ээ 


Observe that r,, is very close to zero, and the last X, vector is very close to the zero vector; if we were 
not rounding intermediate results, they would not exist. However, because of the rounding neither is 
zero, and Q, can be calculated with what are, in effect, error terms. The result is a vector which is not 
orthogonal to either Q, or Q,. 


Supplementary Problems 


20.14 Construct QR decompositions for the following matrices: 


(a) Е 4 i (b) | 4 Э “i (с) | 0 -i | 
- -1 -1 0-1 
2 10 -2-1 0 0-1. 0 
zh ud ө fo -1 0 o] (f) [88 о 90 0 
(a) 5 Eo ES -1 0-1 0 990 660 330 330 
Du. CAL М 0-1 0-1 0 -25 44 -25 
0 0-1 0 0 -880 -8558 0 


In Problems 20.15 through 20.24, use the shifted QR algorithm to determine estimates for all eigenvalues of 
the given matrix. 


[4 42 8 3-2 zc] &@ 
20.15 |4 4 1 20.16 3 S'il 20.17 az 2 =1 
i218 -2-1 4 


2-1 0 0 90 0 99 0 
[42 -131 -86 
э® 2 Sn 0 999 70 330 330 
ui a: ^s 2019 | o -1 2-1 2929 | o -25 550 -25 
0 0-1 2 0 -880 -8,558 110 
[9-2 21 0 1 0 0 1 0-1 0 
2 8-1 0 0 0 1 0 0 2 0 4 
2 | 1 0 Оо! 202 | 9 9 о 1| ?P |5 o9 3 0 
1 1 ol -4 -2 -5 -2 0 -7 0 -4 


Chapter 21 


Generalized Inverses 


PROPERTIES 


The (Moore-Penrose) generalized inverse (or pseudoinverse) of a matrix A, not necessarily 
square, is a matrix А? that satisfies the conditions: 


(11): AA* and A*A are Hermitian. 

(12): AA'A- A. 

(13): A'AA' =A’. 

A generalized inverse exists for every matrix. If A has order л х т, then A* has order m х n and has 
the following properties: 

Property 21.1: — A* is unique. 

Property 21.2: А” = А`' for nonsingular A. 

Property 21.3: (А?) =A. 

Property 21.4: (КА)? = (1/k)A" for k #0. 

Property 21.5: (A")* = (A*)". 

Property 21.6: 0° =0. 

Property 21.7: The rank of A* equals the rank of A. 


Property 21.8: If P and Q are unitary matrices of appropriate orders so that the product PAQ is 
defined, then (PAQ)* = Q"A* P^. 

Property 21.9: If A has order m x К, B has order k x n, and both matrices have rank k, then 
(AB)* = B'A*. 

Property 21.10: For square matrix A, AA” = АА if and only if A* can be expressed as a 
polynomial in A. 


(See Problems 21.13, 21.14, 21.17, and 21.36 to 21.39.) 


A FORMULA FOR GENERALIZED INVERSES 

The following procedure will provide the generalized inverse for any matrix A: 
STEP 21.1: Determine the rank of A, and denote it as K. 
STEP 21.2: Locate a K x K submatrix of A having rank K. 


STEP 21.3: Through a sequence of elementary row and column operations of the first kind (E1), 
move the submatrix identified in Step 21.2 into the upper left portion of A. That is, 
determine 


where P and Q are each the product of elementary matrices of the first kind, and А, is 
a submatrix of A that is nonsingular and of rank K. If no elementary operations were 
necessary, then P and Q are identity matrices. А,,,А,,, or A;; may be empty. 
A a 
STEP 21.4: Set B= [44], F=A,'A,,, and C=[I,|F], where I, is the Кх К identity matrix. 


21 
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STEP 21.5: A* = QC"(CC") (B"B)"'B"P (21.1) 
(See Problems 21.1, 21.2, and 21.16.) When the columns of A form a linearly independent set of 
vectors, (21.1) reduces to 

А? = (АРА) 7А" (21.2) 
(See Problem 21.3.) 


SINGULAR-VALUE DECOMPOSITION 


Equations (21.1) and (21.2) are useful formulas for calculating generalized inverses. However, 
they are not stable when roundoff error is involved, because small errors in the elements of a matrix 
А can result in large errors in the computed elements of A". (See Problem 21.12.) In such situtations 
a better algorithm exists. 

For any matrix A, not necessarily square, the product A"A is normal and has nonnegative 
eigenvalues (see Problems 13.2 and 13.3). The positive square roots of these eigenvalues are the 
singular values of A. Moreover, there exist unitary matrices U and V such that 


(21.3) 


where D is a diagonal matrix having as its main diagonal all the positive singular values of A. The 
block diagonal matrix 


has the same order as A and, therefore, is square only when A is square. 

Equation (21.3) is a singular-value decomposition for A. An algorithm for constructing such a 
decomposition is the following: 

STEP 21.6: Determine the eigenvalues of AYA and a canonical basis of orthonormal eigenvectors 
for A"A. 

STEP 21.7: Construct D as a square diagonal matrix whose diagonal elements are the positive 
singular values of A. 

STEP 21.8: Set V = [V, | V;], where the columns of V, аге the eigenvectors identified in Step 21.6 
that correspond to positive eigenvalues, and the columns of V, are the remaining 
eigenvectors. 

STEP 21.9: Calculate U, = AVD 

STEP 21.10: Augment onto U, the identity matrix having the same number of rows as U,. 

STEP 21.11: Identify those columns of the augmented matrix that form a maximal set of linearly 
independent column vectors, and delete the others. Orthonormalize the columns that 
remain, and denote the resulting matrix as U. 


If A is real, then both U and V may be chosen to be orthogonal. (See Problems 21.4 and 21.5.) 


A STABLE FORMULA FOR THE GENERALIZED INVERSE 
Decomposition (21.3) generates the numerically stable formula 


P 


a=} 


which can be simplified to 
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A’ =V,D U? (21.4) 


where V, and U, are defined by Steps 21.8 and 21.9, respectively. For the purpose of calculating a 
generalized inverse, Steps 21.10 and 21.11 can be ignored. (See Problems 21.6 and 21.7.) 


LEAST-SQUARES SOLUTIONS 


A least-squares solution to a set of simultaneous linear equations AX = B is the vector of smallest 
Euclidean norm that minimizes ||AX — В||,. That vector is 


X-A'B (21.5) 


When A has an inverse, (21.5) reduces to X = A'!B, which is the unique solution. For consistent 
systems (see Chapter 2) that admit infinitely many solutions, (21.5) identifies the solution having 
minimum Euclidean norm. Equation (21.5) also identifies a solution for inconsistent systems, the 
one that is best in the least-squares sense. (See Problems 21.8 through 21.11.) 


Solved Problems 


2 2-2 
A=| 2 2-2 
-2-2 6 


Using Steps 21.1 through 21.5, we first determine that A has rank 2. A 2 x 2 submatrix of A having 
rank 2 is obtained by deleting the second row and second column of A. This submatrix can be moved 
into the upper left position by interchanging the order of the second and third rows and then the second 
and third columns. Then, setting 


21.1 Find the generalized inverse of 


100 2 
P-Q-|0 0 !| givesus PAQ-|-2 
010 2 
2 -2 2 
where hi-[.2 2| ^a |.2] and А, = [2, -2] 
and A,, has rank 2. Then 
2 ~2 
_|-2 6 „аа [6/8 2/81 2].[1 _[1 01 
is ЕТА А = (28 2n] -21- lo] and c-[; 1:0] 
[їз] Tt2 6 нву-_[ 12 -20]" [1/32 5/32 
ю Че) -[ `] -[% i] md (UT al [m $a] 
and A* = QC"(CC")" (BB) ‘BYP 
T 0 ile te opua saj 2 - 3E 0 | 
от оа off 9. 110 5/32 3/3210-2 6 -2]o ро 


3/16 3/16 1/8 
= | 3/16 3/16 1/8 


18 18 1/4 
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21.2 Find the generalized inverse of 
.[0 0 1 2) 
aiii 
The matrix has rank 2. A 2 х 2 submatrix of A having rank 2 (but not the only one) is obtained by 


deleting the second and fourth columns of A. This submatrix can be moved into the upper left position 
by interchanging the order of the second and third columns. Then, setting 


1000 
_fi 0 _{0 010 
Р=[0 1] % Өзу т 
0001 
we get rag=[? 119 All with 


and with both A,, and A,, empty. Then 
ijo 2] [2 -1 T 3] 2 а 
HE 3]- [o Я с-ф 3B 2] 


КЕЕ om emt [ШШ 


01 ы 
B=Au=[1 2] Fes. 


25 “12126 6/26 
and А? «QC"(CC")" (B'B)^'B"P 
100 07 10 
_}9 01 ojj о 1 [226 2/26) 5 EIE n 0 
0100 2 o|l2ne 6261-2 ilt allo 1] 
ооо 11-1 2 
-8/26 5/26 
—16/26 10/26 
2/26 2126 
12/26 –1/26, 


21.3 Find the generalized inverse of 


> 
L] 
ому == юш 


1 
Since the columns of A are linearly independent, we use (21.2). Then 
wem T] эм rmm [2 у, 
wi supuso M pM] 
21.4 Construct a singular-value decomposition for the matrix in Problem 21.1. 


We use Steps 21.6 through 21.11. 


12 12 -20 
$ТЕР 21.6: А“А=| 12 12 -20 
-20 -20 44 


which has eigenvalues 64, 4, and 0 with corresponding orthonormal eigenvectors 


-1У6 1УЗ -1У2 
X, =| -1/V6 X, =| 1/73 X. 1У2 
2/6 1/V3. 0 
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_[ ve 0] f8 0 
STEP 21.7: в-[У& 5]-[8 о] 
-0У6 УЗ -1V7 
STEP 21.8: V|-Uv6 1V3 and У,=| 1/V2 with — V-[(V,[V]] 
2У6 1V3. 0 
2 2 -2]-Uv6 173108 o -1V6 1V3 
STEP 21.9: U, =AV,D" 2 2 -2||-vv6 им [s КАЕ -1У6 1v3 
-2 -2 6] 2/v6 1v3 2IV6 1/3 


STEP 21.10: Augmenting the 3 x 3 identity matrix onto U,, we generate 


-1У6 1У3 1 0 0 
-Uv6 1У3 01 0 
2V6 1У3 00 1 


STEP 21.11: The first three columns of this matrix form a maximal set of linearly independent column 
vectors. Discarding the last two columns and applying the modified Gram-Schmidt 
process to the first three columns, we obtain 


-UV6 1/3 -uv2 
U=|-1/V6 UV3 v2 
2/V6 МЗ 0 


which, in this case, is identical to V. A direct calculation shows that 


воо 
A-v|o 2 oju” 
000 


Construct a singular-value decomposition for the matrix in Problem 21.3. 


Using Steps 21.6 through 21.11, we first form 


which has eigenvalues 28 and 7, with corresponding orthonormal eigenvectors X, — [1,0]" and X, = 

(0, 1]*. Then 

У 0 10 
Qv аа м = [5 1 


У, is empty, so V=V,, and 


-3/У28 1/V7 
-2/У28 VI 


[ ja 0 ]- -ПУ 1/7 


0 1/7 
E O ЖАЙ uV% 1/7 


2/28 17 
2iV28 1/7 


U, = AVD 


o 


We augment U, on the right by the 7 x 7 identity matrix and then determine that the first seven columns 
of the augmented matrix form a maximal set of linearly independent column vectors. We discard the last 
two columns of the augmented identity matrix and apply the modified Gram-Schmidt process to what 
remains, generating 
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3/28 107 15/420 0 0 0 0 
-2N/28 1V7 -10/V420 10/V210 0 0 0 


-0У28 107 -7N42) -8/'v210 2V0 0 0 

0 1/7 -4/V4200 -5/V210 -2V0 3/30 0 
1/V28 1/V7 -1/420 -2/210 -1/V10 -4/vV30 1/6 
2V% 1V7 27400  l/v210 0  -1/V3) -2/V6 
3/28 1/V7 57400 4/210 1У10 2/30 1v5, 


A direct calculation shows that 


U- 


v28 


v 


ооооо%о 


сооооо 


21.6 Use (21.4) to calculate the generalized inverse of the matrix in Problem 21.1. 


Using what we have already found in Problem 21.4, we compute 

Ата -0У6 INS) 11g o -1V6 -1v6 2N$]. 3/16 3/16 1/8 

АМО! -1V6 VIVE о vall туз цуз 1313/16 3/16 1/8 
2/У6 1173. 1/8 1/8 1/4 


21.7 Use (21.4) to calculate the generalized inverse of the matrix in Problem 21.3. 
Using what we have already found in Problem 21.5, we compute 
А? =%,07'0" 


Е [а -3/V28 -2/V28 -1У28 0 1У72 21V28 3/VB 
oal o 


0 
uv] 17 aO їМ? ANO avi AIT. AINT 


_[-3/28 -2/28 -1/28 0 1/28 2/28 328] 
Lam 1/7 01/7 17 in ш 1/7 


21.8 Solve the following system of equations in the least-squares sense: 
2x, +2x,-2x,=1 
2x, + 2x, —2x,;=3 
—2x, — 2x, + 6x,=2 
This system is inconsistent. Writing it in matrix form and then using (21.5) and the results of either 
Problem 21.1 or Problem 21.6, we obtain 


3/16 3/16 1/8ļf1 1 
X=/3/16 3/16 U/8|3|-|1 
1/8 1/8 1/4302. H 


Therefore, x, = з = 1 is the solution in the least-squares sense. 


21.9 Solve the following system of equations in the least-squares sense: 


x,*2x,-71 
X, +2x, +2x, +3x,=2 


Writing this system in matrix form and then using (21.5) and the results of Problem 21.2, we obtain 
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-8/26 5/26 113 

x- -16/26 10/26 [3]- 23 
2206 20642 3/13 

12/26 -1/26 5/13, 


Thus, x, = 1/13, x, = 2/13, x, = 3/13, and x, = 5/13. 


Verify that the solution obtained іп Problem 21.9 is the solution of minimum Euclidean norm 
for the set of equations given in that problem. 


Interchanging the order of the two equations, we obtain the system 
x, +2x, +2x, +3x,=2 
ху+2х,=1 


whose coefficient matrix is іп row-echelon form. Using the techniques of Chapter 2, we determine the 
solution to be 


X=] 241 a) 


with x, and x, arbitrary. For this vector, 
IXI = (72x, + x4)? + xi (72x, + 1 + xi = 5x2 + 6x3 c xx, - 4x, 41 
The minimum of this function occurs at its critical points. Setting the first partial derivatives equal to 
zero, we get 
10x, – 4x, =0 
-4x, + 12x,=4 


which has the unique solution x, = 2/13 and x, = 5/13. Substituting these values in (1) gives us the same 
solution as we obtained in Problem 21.9. 


Sales data were taken over a seven-year period of time and coded so that the midpoint of the 
period coincides with time / = 0. The results are given in the following table (in hundreds of 
thousands of dollars): 


A graph of these data reveals a near linear type of growth. Find the equation of the straight 
line that best fits the data in the least-squares sense. 


A straight line here would satisfy the equation 
S=att+b a) 


where 5 denotes sales, ! denotes time, and a and b are constants to be determined. Substituting each 
data pair into Eq. (1) yields the system: 


-3a*b-10 
-2a* b-15 
-a+b=19 
b=27 
at+b=28 
2а+Ь=34 


За+Ь=42 


CHAP. 


21.12 


21.13 


21.14 
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Writing this system in matrix form, and then using (21.5) and the results of either Problem 21.3 or 
Problem 21.7, we obtain 


хе [4] [73/2 —-2/28 -1/28 0 1/28 2/28 san] 27 - [1825] 


b 1/7 17 1/7 A7 WI 1/7 1/7 25 
42 


The equation of the line that best fits the data in the least-squares sense is 5 = Mr + 25. 


Working to four significant digits, show that (21.2) is numerically unstable when applied to 


11 
A-|11 
1 1.004 


Rounding all stored (intermediate) numerical quantities to four significant digits, we calculate: 


ava = [300 3.004) so det AYA = —1.600 x 107 


3.004 3.008 
TAN 1 3.008 -3.004 = |08000 ied 
Then (АА) --v$x1071-3004 3.000 1.878 х 10° —L875 x 10° 
2 cea wena 5:200 200 s] 
and a = (АНАУ A = | 300 aao 


This compares unfavorably with the actual generalized inverse 


[8° ws” am] 


calculated without rounding off. Similarly poor results are obtained when results are rounded to k digits 
and the component 1.004 is replaced with the more general 1 + 4 x 107***. 


Show that the generalized inverse is unique. 


We assume that D and E are two generalized inverses for the same matrix A and then show that 
D = E. Since both D and E are assumed to satisfy conditions 11, 12, and 13 we know that DA, AD, EA, 
and AE are all Hermitian and that 


ADA=A DAD=D AEA=A EAE=E 


Multiplying the first of these equations on the right by E, we obtain ADAE = AE, from which we infer 
that 


AE = (AE)" = {(AD)(AE)}” = (AE)"(AD)" = (AE)(AD) = (AEA)D = AD 

Multiplying that same equation on the left by E, we obtain EADA = EA, from which we deduce that 
EA = (EA)" = ((EA)(DA))" = (DA)"(EA)" = (DA)(EA) = D(AEA) = DA 

Then Е = EAE = (EA)E = (DA)E = D(AE) = DAD = D 


Show that if P and Q are unitary matrices of appropriate order so that the product PAQ is 
defined, then (PAQ)' = Q"A* P^. 


Let G = PAQ. We need to show that G* = Q"A*P" satisfies conditions 11, 12, and 13, given that A 
and А” do. 
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IH: СС” -(PAQ)(Q"A' P) = PA(QQ”)A’ P" = PAIA’ P" = P(AA’ )P” 
G'G = (Q"A' P^)(PAQ) = Q"A' (P"P)AQ = Q"A' IAQ = Q"(A'A)Q 
Both are Hermitian since АА” and A'A are. 
12: GG'G = (PAQ)(Q"A* P^)(PAQ) = PA(QQ”)A’ (P"P)AQ = PAIA* IAQ = P(AA*A)Q = PAQ = G 
B: G'GG' = (Q"A' P^)(PAQ)(Q"A' Р”) = Q"A' (P"P)A(QQ")A' P" = Q"A"IAlA' P 
= Q"(A'AA*)P" = Q"A' P" =G* 


Show that if A can be factored into the product BC, where both B"B and CC" are invertible, 
then A* = C'(CC") (BB) 'B". 
We need to show that A* satisfies the three conditions required of a generalized inverse. 
її: AA‘ = (BC)C"(CC"); (B"B)'B" = B(CC")(CC") '(B"B) 'B" = B(B"B) 'B" 
A'A = C"(CC")^(B"B) 'B"(BC) = C"(CC") '(B"B) '(B"B)C = C"(CC")''C 
Both are obviously Hermitian. 
D: AA'A-(BC)C"(CC") '(B"B) 'B"(BC) = B(CC^)(CC") ']((B"B)"'(B^B))C = BIIC = BC = A 
13: A'AA' = СМСС") (BB) 'B'(BC)C"(CC") (B"B) в" 
= C"(CC") '[(B"B)^ (B^B)I(CC")(CC") 'в“в) 'в" 
= C"(CC")- BB) "B^ = C'(CC") (B^B) "B" = А” 


Validate the algorithm given by Steps 21.1 through 21.5. 


Steps 21.1 through 21.4 provide a procedure for factoring the matrix PAQ into the product BC, 
where both B"B and CC" are nonsingular. Observe that the last n — K columns of PAQ are linear 
combinations of the first К columns, so there must exist a matrix Е such that both A;, = А, Е and 
A, = AF. Since A,, is invertible, F = A;,'A,, and A,, = A,,A;,'A,2. Now it follows from Problem 21.15 
that (PAQ)' = C"(CC")" '(B"B) 'B", and then from Problem 21.14 that 

Q"A' P" = C"(CC")" (B"B) B^ 
The desired formula comes from multiplying both sides of this last equation by Q on the left and P on 
the right. 

Although the factors B and C and the matrices P and Q are not unique, the product is, as a result of 
Problem 21.13. 


Prove that (A")* = (А*)”. 
Set G = A". We need to show that G* = (A*)" satisfies conditions 11, I2, and I3. 
її: (GG')" = (A"(A' )")" = ATA = (A'A)" = ANAT)” = GG* 
(G°G)" = (A A" = AAT =(AA") = (A A" =G*°G 
Both are Hermitian by the definition (13.2). 
D: G'GG' -(A')"A"(A*)" = (ААА) = (А) = С' 
13: GG'G = A"(A' )"A" =(AA‘A)” = A" -G 


Thus, G’ = (A' )” satisfies all the conditions for a generalized inverse for A", and since the generalized 
inverse is unique, it follows that (A")' = G' =(A")*. 
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Supplementary Problems 


In Problems 21.18 through 21.24, find the generalized inverse of the given matrix, 


tid 
1 113 Е 
21.18 E] 21.19 E 1 2| 21.20 [ 1 | 21.2 12 
11 141 
za |20| 2123 |3 23 21] жй f 1 | 
01 123 


In Problems 21.25 through 21.38, find the least-squares solution to the given system of equations. 


21.25 x, +x, +3x,=1 21.26 x, +x,+x,=1 
x, +x, 43x, =2 xo 4%, 42,52 
X,+2%,42,=3 


21.27 x,*x,-1 2128 х, +2х,+3х, 4ox,Q-1 
2x, =2 -х,+2 x,72x,*3x,71 
3173 


21.29 Show that the least-squares solution of AX = B must necessarily satisfy the system A"AX = АҮВ (the 
normal equations for the original system). 


21.30 Show that if А = ОК is a QR decomposition of A, then the normal equations can be written as 
R"RX = R"Q"B, which reduces to 
RX -Q"B a) 


when the columns of A are linearly independent. 


21.31 A numerically stable procedure for determining the least-squares solution to the matrix system AX = В 
when the columns of A are linearly independent is to first determine the QR decomposition for A"A and 
then solve (1) of Problem 21.30 for X. Use this procedure to solve Problem 21.11. 


21.32 Use the procedure described in Problem 21.31 to solve Problem 21.27. 


21.33 The following data appear to be quadratic when graphed: 
x Sb 3 


10 14 18 32 49 


Determine the normal equations for the quadratic equation, y — ax^ bx ^ c, that best fits these data in 
the least-squares sense (see Problem 21.29). Solve for a, b, and c using the procedure described in 
Problem 21.31. 


21.34 Construct a singular-value decomposition for 


nara 
a[i 1 3 
21.35 Construct a singular-value decomposition for 
7 
^-[; 


21.36 Prove that А’ = A^! when A is nonsingular. 
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21.37 


21.38 


21,39 


21.40 


21.41 


21.42 


21.43 


21.44 


21.45 


21.46 


21.47 


21.48 


21.49 


21.50 
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Prove that 0° = 0. 

Prove that (А’)' =A. 

Prove that (kA)* = (1/K)A', provided k #0. 

Prove that if A is Hermitian, then so too is А“. 

Prove that if A is Hermitian and idempotent, then A* = A. 
Prove that AA* and A*A are Hermitian and idempotent. 


Show that if A has order m х n with m = n, then A can be factored into A= ОХУ", where X is ann x n 
diagonal matrix, V is an n X n unitary matrix, and U is an m X n matrix with orthonormal columns. 


Using the matrices identified in Problem 21.43, show that P - VEV" is positive semidefinite, and 
M = UV" is a matrix with orthonormal columns. 


Using the results of Problems 21.43 and 21.44, show that any m X n matrix A with m = n can be factored 
into А = MP, where M has orthonormal columns and P is positive semidefinite. Such a factorization is 
called a polar decomposition of A. 

Find a polar decomposition for the matrix in Problem 21.1. 


Find a polar decomposition for the matrix in Problem 21.3. 


Show that the positive semidefinite matrix P defined in Problem 21.44 as part of the polar decomposition 
of A can be given by P= VA"A. 


Show that if A is invertible, then the matrix M defined in Problem 21.44 as part of the polar 
decomposition of A reduces to M = (PA !)" = (А) 'Р. 


Use the results of Problems 21.48 and 21.49 to determine a polar decomposition for 


TE 


Answers to Supplementary Problems 


CHAPTER 1 


=z 9 -12 -14 -6 -7 - 


119 (a) [1 4] (b) [s 6) (c) IE a] (4) Е E il (e) undefined 


120 (а) -10; (b) 23 (с) -1 
121 (a) [ 2 $] (b) [5 28] (c) (2 -4 (4) [2 aj (e) [ 6 7] 
2; 7 -6 6 -2 6 -2 -8 -6 


-10 -10 -5 6 12 6 -13 20 


122 (a) | 26 26 Hd (b) [ 6 d 1.23 | 9 0 
16 l6 8 9 18 9 


-5 


124 (a) [5] (b) undefined 1.25 (a) undefined; (b) [2, 5, 5) 
8 


246 0 


126 (а) Е 2 Д (b [4] 127 [! 2] 
1 
369 


1.28 [i 2] 129 [1 5/3 0 
01 D t 3 
ооо 


1.30 [1 1 1/2 131 [1 
00 0 0 
00 0 0 


132 (а) 2; (b) à; (с) 2; (d 1; (е) 1 
1.33 There are many examples; one is 
12 é -2 
НЕ 
1.34 Total sales revenue for the flight 
1.35 ND = (20700, 15300, 4900]’, a vector representing the money invested by each store in unsold computers 
of both brands 
CHAPTER 2 


2.15 (b) and (c) are solutions. 


216 [13 1:5] 217 [2-4 7 6 4117 
2.1 -3 115 0 6 -3 -4 -512 
їз 35d 2 8 1 -2 -14!10 


248 (а) x,=39,x,=5,x,=-4; (b) x, =-7—3x,, x, arbitrary, x, =5, x, =0 
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219 x, =8+2x,, x, 7 —1- x,, x, arbitrary 
2.20 The system is not consistent. 


2.21 Only the trivial solution x, = x. 0 exists. 


222 x,74-*x, x, = -2х,, x, arbitrary 

223 x, =8, 52-2, 3,7 -3 

2124 x =x, =4,=2,x,=-1 

225 x, -560, x, = 4860, x, = —10920, x, = 7000 


2.06 x, = –19999.302022, х, = —9999.298601, x, = 0.690113, x, = 10000.707204. The system is inconsistent 
when results are continually rounded to four significant figures. 


2.22. x, = 0.49998, x, = 0.00001, x, = 0.5 


2.28 (a) Consistent only if k = 7, and then x, =2—x,, x, = 1 + x,, x, arbitrary; (b) consistent only if k = 2, 
and then x, = x, = 2. 


2.29 There are infinitely many solutions to the system 


12c + 10d + 6r = 440 
5с+ 34+ r=120 


2.39 b- 0.03 +0.03с + 0.03s =0 
00li- c =0 
0.041 -0.04c- — s-0 

231 A(Y-Z)=AY-AZ=B-B=0 


2.32 Y - Z is a solution of the homogeneous system; simply call that difference H. 


CHAPTER 3 
313 (а) [1 0 0 (6) [7 0 0 (с) 100 
oot 010 -310 
010 001 001 
3.14 (а) [10 0 0 (b) 1000 (о [10 5 0 
0010 0100 0100 
0100 0010 0010 
0001 -3001 0001 
3.15 (a) È 0 (5) [52 1 ] зав |[-1-4 2 
3/2 -1/2 12 -1/2 51-6 1 2 
3 2 -1 
3.17 Let A and B be n X n lower triangular matrices. Then a,, = 0 if k >i, and b,, =0 if j> К. Set C= AB. 


For j >i, 


+ $ aud, =D a,(0)+ Ў (0b, =0 
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3.18 [1 оо 2 3 3.19 2 0 0 l2. 2 
4.3 0 |2: 17/2 0 0 1 27 
7 -6 0 01 -1 3/2 11/7][0 0 1 


3.20 


coon 


1 
2 
0 
0 


3.24 The factorization cannot be done. 


322[1 0 0 0 1253 4 
4 5 0 0 0 1 7/6 7/6 
1-5 41/6 0 00 1 -13/41 
2 -2 -20/3 -155/41JL0 0 0 1 


323 х,=х,=1,х,=2‚,х‚,=-2 324 х,=4, х,=0, х,=1, х,=0 
3.25 x, =0, х, =х;=1, х= -1 3.26 х,=4+у;, x, = -2y, x, = y, = y; уз arbitrary 
3.27 Cannot be solved; LY = В is inconsistent. 328 x, =8, х,=-2, 2, =-3 
329 x,=2x,=2x,=2,x,=-1 
sala] eal [stg 
00 -1 
3.32 Each diagonal element is raised to the pth power. 
3.34 (I-A = (L- A(I- A) -F - IA- AI- A! - IC A-A* A- I-A 


3.35 By induction: The proposition is obviously true for p = 1. If it is true for p = k — 1, then 
(A^) = (ASA) АТАЎ 1) = (AT) (AP = (AT) 


CHAPTER 4 


4.15 These are elementary matrices: (а) the matrix is its own inverse; (Б) change 7 to 1/7; (c) change 4 to —4; 
(d) change —3 to 3. 


4.16 [ 1/2 ?] 4.7 I3 2) 

-1/2 1 2 = 
4.18 Noinverse 4.19 1 [i 4 
ML, Җ 

4.20 10 0 421 [1 -1 1/5 

-1 1/2 0 о =L 3/5 

13 0 13 0 0 1/5 


4.22 No inverse 


4.23 1 =f 4 7 
|6 4 -14 
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7/2 1/2. 


17 4.25 1 -3/2 12 
-8 1 Fh 0 
32 


4.26 -26 11 2 i 


4.27 x, = 37/28, x, = —26/28, x, = ~44/28 
428 х,=3,х,=0,х,=-3 429 x, = -9/2, x, 71/2, x, 5/2, x,- 1/2 


4.30 (A')'' is the inverse of А”, Now A'(A^!)' = (А 'А) = [7 =1, so (A^')' is also the inverse of A^. 
Equality follows from the uniqueness of the inverse. 


4.34 Each part follows from the uniqueness of the inverse: (а) A^'B^! and B^'A'^' are both inverses of AB; 
(b) А !В and BA™' are both inverses of AB ^'; (c) AB^' and B^'A are both inverses of BA ^". 
CHAPTER 5 
$.21 (a) -3; (b) —33; (c) -2916+ 3015= -3(-33)- 99 
$22 (a) 0; (b) -28 (c) 0-0 
5.23 (a) -48; (b) detF is undefined because Е is not square. 


5.24 (a) -1L1 *6] (b) 1[-6 3] 


3 7331-5 8 
x EE =, p iu -в| 0 16 "s 
Wles 4 0 48 [18 -16 n 
527 0 528 2 
5.29 -26 11 Ж -1 


5.30 Denote the equal rows as rows i and j. Add —1 times row i to row j, and then use Properties 5.5 and 5.6. 
5.31 Use Property 5.4 n times, once on each row of A. 

5.32 det A? = det AA = det A det A 

5.33 det AB = det A det B = det B det A = det BA 

5.34 1 =detI = det AA™' = det А det A ;sodet A™' = I/det A. 


5.35 Note that det A = det LU = det L det U, and then use Property 5.2. In particular, det U = 1 since it has 
only unity elements on its main diagonal. 
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CHAPTER 6 


6.15 
6.17 
6.19 
621 
622 


6.23 


6.24 


6.25 


627 


6.28 


6.29 


6.30 


631 


6.32 


6.34 


Linearly dependent 6.16 Linearly dependent 

Linearly independent 6.18 Linearly dependent 

Linearly independent 6.20 Linearly dependent 

No 

(a) Yes, [0,0,1] = O[1, 1, 2] + 1[2, 2, 2] + (-1)[2, 2, 1]; (b) no 


(а) Yes, [2, 1,2, 1] = 2[2,0, 1, 1]  (- D[0, 1,2, -1]+ (—2)[1, 1, —1, 1] + 0[0, 0, 1,2]; 
(b) Yes, {0, 0, 0, 1] = (1/3){2, 0, 1, 1] + (-2/3)0, 1, 2, —1] + (—2/3)[1, -1, —1, 1] + (1/3)[0, 0, 1,2] 


-2a*b*2c 


-2b4 *2b- 
4a-2b*c a+2b * (1,2,0] 4 : (0. 1,2] 


[а,Ь,с] = 5 [1,0, 1] + 5 


(2) — 6.26 {[1, 1,2], [2, 2, 2)} 
{{1, 2, t, —1], [1, 0, –1, 2), (0, 1, 1, 0) 


[5/3, 5/6] = (1/2)[1, 1] + (1/3)[3, 0] + (1/6)[1, 2] 
al-s s ES eti] t1) 
PEHHLETH EE HE IP 
No vector in the set (V,, V}, .. . , V,, V) can be written as a linear combination of vectors preceding it, 
not the first r vectors (as a result of Property 6.2), because they are linearly independent, and not the 


last vector in the set, because of the hypothesis. It then follows from Property 6.2 that the entire set is 
linearly independent. 


Consider (V,, V,,..., Va, 0). Then c, 7c; =+ = с„ =0,c,,, = 1 is a set of constants not all zero such 
that c,V, + с, +++ cV, + (10) = 0. 


ому [9] ex [75] - [73] wit x, arbitrary 


=x jtt 


=2 
RS = 0 | with x, and x, arbitrary 
1 


CHAPTER 7 


All x, in the solutions for this chapter denote arbitrary constants. 


7.18 


7.19 


7.20 


x[ 71] for A=0 and АН for A=4 


alt] for A= 1, an eigenvalue of multiplicity two 


x{ 4] for A=3 and x[?] for A=9 
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721 


722 


7.23 


7.24 


7.25 


7.26 


7.27 


7.28 


7.30 


7.33 


7.34 


7.35 
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[12] for a=6 and akal for à= -6 
x [69-9] for A = УЗ and xf 2992] for A= -V$ 


-1-i2 
l 


1 1 0 
x|0| ford=1, АН for A=2, and 2H for A=3 
0 0 1 


1 0 24, 
ZH * ZH for A = 0 (of multiplicity two) and 4 | forA--1 
0 


] for A-2- i2 and ыы for A=2-i2 


1 0 
-2 =з 1/3 
x| l|*x| 0| — for A= —3 (of multiplicity two) and — x,2/3] — forA=11 
0 1 1 
1 
0 for А = 5 (of multiplicity three) 
0 


1 0 
oj and [o] for А = 5 (of multiplicity three) 
0 

] 


0 0 
H and [| for А = 5 (of multiplicity three) 


1 
| and for A=3 (of multiplicity four) 
0 


1 
h and | í| for A=1 (of multiplicity three) and 
0 


1 1 
| = | and H for eigenvalues 2, 3, and 6, respectively 
1 1 


1 1 1 
] В апі [=] for eigenvalues 12, 6, and 4, respectively 
-2 1 0 


(-1.1] and (1, 3], corresponding to eigenvalues 0 and 4, respectively 


7.36 


7.37 


7.38 


7.39 


7.40 


7.42 


7.43 


7.44 


7.45 


7.46 


7.7 


7.48 


7.49 


7.50 
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[71.1], corresponding to the eigenvalue 1 (of multiplicity two) 

[1, -2] and [1, 4], corresponding to eigenvalues 3 and 9, respectively 

[1, 1, 0], [1,1,2], and (1, 1, —1], corresponding to eigenvalues 1, 3, and 6, respectively 
[71, 1, 0], [1, 0, 1], and [1, 1, -1], corresponding to eigenvalues 0, 0, and 3, respectively 
[1,1,0], [71, 0, 1], and [1, —1, 1], corresponding to eigenvalues 2, 2, and 5, respectively 
А?Х = A(AX) = A(AX) = A(AX) = A(AX) = А?Х 

(А - cDX = AX - cIX = AX - cX = (А – с)Х 


The proof is by induction on the order of the matrices. The proposition is certainly true for 1x 1 
matrices. Assume it is true for k X k matrices, and let A be an arbitrary (k +1) х (k * 1) matrix. 
Designate as A’ the matrix obtained from A by deleting its first row and column. Then A’ has order 
kx k, and the induction hypothesis can be used on it. Evaluating det (A — AI) by expansion of the first 
row, we get 


det (A — AI) = (a,, — A) det (A’ — al’) + O(a*~?) 

= (à, — А)(—1)*{А* ~ (trace A')A* + O(A*7)) + O(A* 7) (by induction) 

7 (- ^ (A**' = (a,, + trace A')A* + О(А 1) 

= (-**(a**! - (trace А)А* + O(A*7)) 
Denote the eigenvalues of A as A,, A. -s Àn. Then 
det (A — AI) = (C1)'(A - A, )(A = Ag) (A7 Ap) = (CD {A" = (A, + Aj ++ + ADAT! O(A"™)} 
But from Problem 7.44, 

det (A — AI) = (—1)"{A” - (trace A)À" ' + O(A"7)) 


The result follows from equating the coefficients of A^^' in the two expressions for the characteristic 
polynomial. 


trace (A + B) = (а, + b) + (а, + by.) +++ + (а, + Onn) 
= (0. +05 +++ ann) + (By + baz tas) 
= trace A + trace B. 


trace AB = x (È DON - x ( $ buan) = trace BA 


M iN 
Using the results of Problem 7.47, we have 
trace 5 'AS = trace{S~'(AS)} = trace((AS)S ') = trace(A(SS" ')) = trace AL = trace A 


Denote the eigenvalues of A as Ay, Az,.-., А„. Then det(A — AI) = (-1)'(A - A, (A= A) (A - An). 
Set A=0, and det A = АА-А. 


The proof is by induction on the order of the matrix. When C has order (k + 1) x (К +1), expand 
det(C — AI) along the first column, obtaining the sum of two determinants. Use the induction hypothesis 
on the cofactor matrix of the element —A in the (1,1) position. The second determinant is easy to 
evaluate because the cofactor matrix of — a, is lower triangular. 
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CHAPTER 8 
Я ДА i] улы у. 
a ima [o | mef 0 0] 
lim C, does not exist because lim ((k — Kk 1) = -. 
8.20 Every square matrix A 
8.21 All eigenvalues must be less than 4 in absolute value. 
aL l[4sins-*2sin(-1) 2sinS-2sin(-1)] [-0919773 —0.039151 
8.22 (a) йад 2 [ens dan 2sin 5 + 4sin(-1) = [2291972 E 
(b) eres 26-26] [99096 49.3484 
6l4eé -4e 26 + дет! 98.6969 49.7163 
.l[4cos1-2cos(-1) -2cos1+2cos(~1)] [0.540302 0 
8.23 (a) esa 1 [aen re Cl) icy lal 0 m 
yal 76 
9) 3a7+2a"=[1) $] 
8.24 (a) sinA=0; (b) cosA-I 
8.25 Ie 0 8.26 [ 3e – 2е-' заз 
0 е” -2é *2e€" -2е'+3е' 
8.27 [$ ©] 8.28 [етан 3 sin 2¢ ] 
о e -2sin20 — cos2t—2sin2t 
8.29 ) or 02 8.30 100 
e"l01 ¢ tO Lt 
00 1 001 
8.31 ,[9&' -3e +3e” е-е 31] 832 [sind 0 0 [] 
| 9 9e” Яге? 0  sin2t tcos2r 0 
0 0 9e" 0 0 sinu 0 
о 0 0 sin3r 
[ger get 
заз X()=| den dern 
8.34 C - [с,, c;]" is arbitrary. The solution vector, in terms of C, is 
x)= (3c, +e, + фу (с, AEN s H 
((79c, + Зс, + Dre (с, 7 je" +4 
_ [hen 4 de" à Гре зе 
8.35 xo-[ aeger 836 X()- |. еу 8.37 X 
CHAPTER 9 
9.16 (c) and (d) 
9.7. X., X, = [-1, 0, 2, 1, 0]7, X, = [2,0, —1,0,0]", X, =[-1,0,0,0, 0)” 


9.18 


9.19 


9.20 


9.21 


9.22 


9.23 


9.24 


9.25 


9.26 


9.27 


9.28 


9.29 


9.30 


9.31 


9.32 
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{0, 1,0)” 

X,, X, = [0, 1, 0]", X, =[1, 0, 0]” 

(a) Two chains of length 2; (Б) one chain of length 2 and two chains of length 1; (с) one chain of length 3 
and one chain of length 1; (d) two chains of length 3 and one chain of length 2; (e) one chain of length 
3, one chain of length 2, and three chains of length 1; ( f) three chains of length 2 and two chains of 
length 1; (g) the eigenvalue rank numbers as given are impossible. 


(a) N,=N,=N,=1 and N,=2 for A=1; (b) the vectors found in Problem 9.17, along with 
Y, 7 [0, 1,0, 0,0)” 


(а) N,=N,=N,=1 for A=5; (Б) the vectors found in Problem 9.19 

(а) N,=N,=1 for A=-2; (b) X,-[0,1]", X, = [-3,3]" 

(а) N,=1 for both A-OandA-4; (b) X,=[-3,1]’, Y, - [1, 1]" 

(à) N,=1,N,=2 for A=2; (b) X,-[0,0, 1]", X, = [1,0,0]", Y, - [0, —1, 2]" 

(a) N,=2 for A=4 and №, =1 for à=-1; (b) X, -[1,0, 1], Y, = [1,1,0]", Z, 2 [-2,0,3]" 
(а) №=1апі № =2 for A=1; (b) X,-[0,0, 1]", X, = [1,1, 2)", Y, 2 [-2, 1, 1]" 

(а) N,=N,=1 for A=2 and №, = 1 for A=4; (b) X; = [0,1,1], X, = [4,4, -8]", Y, = [-1,0, 1] 


(а) N,=N,=1 for both A=2 and A=3; (b) X,-[0,0, 1,0]", X, = [1,0, —1, 1]", 
Y, =[-5, 71,2, 0]", Y, =[0,0,0, 1)” 


(а) N,=N,=1 for A=2 and №, =2 for A=3; (b) X,-[0,0, 1,0]", X, = [1,0, —1, 1]", 
Y, =(0,0, 0, 1)”, Z, =[-3, 71, 1,0)” 


(а) №= №, =1апі N,=2ford=5,N,=1forA=2; (b) X,-0,0,1,0,0]", X, = [2, —1,0,0, 0], 
X, =[-1,0,0, 0,0], Y, = (0,0,0, 1, 1], Z, =[0, 0,0, -2, 1)” 


(а) (A= 2; (b) (A= 4)(A + 1); (с) (A — 1; (d) (А—2) (А - 4); (е) (А - 2) (A - 3; 
(Р) (А 2/(4 73) 


CHAPTER 10 


10.20 


10.22 


None are similar. 10.21 [3 "i 
31 


a ited 
2 
.25 


Lod 2 10. 
01 0 
i05 3 T 


m 

о-оо 

-ooo 
D 
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————— 
ocoooo-a 


ooooooao 
oocoo-aoo 
ooo-caocoo 
ооочоооо 
оочооооо 
-чоооооо 
чооооооо 


(а) 


a 
coooooo-d 


oocoooonao 
oocoocacc 
oooonacoo 
oo-aoooo 
oocaocoocooo 
oncnocooooo 


n noocooooo 
[a на А 


(f) 


——— вру: ) 
оооооо- еч 


oocooooao 
оооо- чоо 
роон чооо 
oooacooo 
oonacoooo 
onocooooo 


ссооооооо 
а 


= 


gg 


-2 
0 


qma f 


aaa 
a 
= 
оов 5 
E 
1 
San 
oo 888 
858° 
wy 
i 
ag 
оо Ёоо 
Т 
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2 0 0 e 
0 0 2 


10.39 [ 3e'-2e* sae 10.40 100 
e"o1:t 


10.37 [ 62 "| 10.38 [© 6] 
2 


—2e «2e! -2e + 3e7 


10.41 р [967 -3e +3e” e" - e" ет 
0 9e te" 
0 0 9e” 
10.42 Premultiply (10.1) on the left by S; then postmultiply on the right by S^' and set Т=5 7. 


10.43 Premultiply (10.1) on the left by S to obtain SA = BS. Then 
BY = B(SX) = (BS)X = (SA)X = S(AX) = S(AX) = A(SX) = AY 


CHAPTER 11 
1113. (a) 0; (Б) 0; (c) 3; (d) 2; (е) 14; (f) —6; (8) 2 

1114. (a) 144; (b) 1- i; (с) 4- i2; (d) -1- i; (е) iS; (f) 50- 125 
1.15 Q, =(1/V2)(1, 1,0, = (1V2)[-1, 1)" 

1116. Q, = (1/V13)[3, 217, Q, = (1/ү6;877)[46, —69]" 


11.17 The given vectors are not linearly independent. The Gram-Schmidt process produces Y; = 0, on which 
Step 11.4 cannot be performed. 


1118. Q, = (/V2)[i 1], Q, = (172) - i, 1 + i)” 

11.19 ©, = (1/У2)[1,1,0]", ©, = (1/V6)[71, 1,2], ©, = (1/УЗ)у[1, 71, 1]* 

11.20 Q, =(1/V6)[2, 1, - 1], О, = (1/V29T)[- 1, 13, 11)", ©, = (1/У7194)[8, -7, 917 

11.21 Q, = QUV3)[L, 1,1)”, о, = (1/\/$)[1, 1, —2]7, ©, = (1/V2)[ 71, 1,0]* 

11.22 Q, = (1/У2)11.0, 0", ©, = (УТ)  i2,2.2 - i] Q, = (LV 35)[-2 = i, 4 - i3, -1 + 2)" 

11.23 Q, = (1/V2)IL, 1, 0, 0]", Q, = (1/V6)[=1, 1, -2,0]*. ©, = (1/V3)[1, 71, -1,0]", ©, = [0,0,0, - 1]^ 


11.24 Q, = (1/V2)[1, i, 0, 0]", Q, = (t/V18)[, 1, 4,0]", ©, = (1/VT17)[i4, 4, -2, 9]. 
Q = (1/V13)[2, - i2, i, i2]* 


1125. Q, =(1/V10)[1, 0]. ©, = (1/V10)[-7, 5] 
11.26 Q, - (1/V8)[1, 1, 0], ©, = (1/V23)[-1, 1, 2)". ©, = (1/М/З)[1, 71, 17 


11.27 Consider the equation c,Q, + с,0, ++- + c, Q, = 0, where ће Q, vectors form an orthogonal set. For 
each i (2 1,2,...,n), 
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0-0,0), (Ў «0. 9), - È оо) = «(оь о) 


i= 


Since Q, is nonzero, so too is (Q,, Q,) w; thus, c, =0. 


11.28 Set WX=Y=[y,, ys... Ya] Then (X X)y = (МХ): (WX) = 
(X, X), 20. Furthermore, if X#0, then Y 2 W^'X»*0, and Z7. , |у, 


SY REL. pF Therefore, 
is positive. 


11.29 If we continue on Problem 11.28, it follows that (X, X),, = 0 if and only if Y = 0 and that is the case if 
and only if X 2 W^'Y - 0. 


CHAPTER 12 

12.19. (a) 1; (b) УТД; (c) УЭТ; (d) У65; (е) VB 

12.20 (а) 1; (b) 6; (c) 7; (4) 9; (е) 12 

12.21 (а) 1; (b) 3; (с) 4; (4) 8 (е) 4 

12.22 (а) УТО; (b) V4; (c) V; (d) V298; (e) У464 

12.23 (a) 3"*; (b) УЗ; (c) 3; (4) (155); (е) 11; (f) 4 

12.24 (a) 1; (b) V; (с) 8; (4) V 

12.25 (a) 1; (b) 7; (c) 8; (4) 5+ VS 

12.26 (а) 1; (b) 5; (с) 8; (4) 5 

12,27 (a) 1; (b) 133'^; (c) 8; (d) 5.1448 

12.28 (a) V30; (b) У79; (c) VOB; (4) VZ; (e) У75 

12.29 (a) 6; (b) 12; (c) 9; (d) 5; (е) 8.9443 

12.30 (a) 7; (b) 8; (c) 9; (d) 5; (e) 9.4721 

12.31 (a) 5.4650; (b) 8.6099; (c) 8.1231; (d) 4.2992; (е) 7.1517 
12.32 For any induced norm, ||} = max,x,. (Х|) = maxyxy-, (IXI) = 1. 


12.33 Denote the ith row of A as A,, and the jth column of B as B,. Then 


jani} = Š È 


"2S Da apt 


ia 


= [NM 
m 
and, by the Schwarz inequality, 
lasis È È (A1, AD (B, B.) 
me 


=> 3( 


fied ‘ke 


teal? E lon) = (5, È lea?) (3 È wr) = tato 
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1234 [X +Y = (X * Y.X + Y), = (XX) + (X. Yow + (Ү,Х)„ + (Y. Y) 
206 X). + Y), = XIS + YI 


12.35 (a) V30; (b) 8; (с) V66; (4) VZ; (e) V75 
12.36 (a) 5.3723; (b) 6.7958; (c) 8.1231 

12.37 4.8990 

12.38 The eigenvalues of A and A” are identical. 
12.39 (a) 15; (b) 4.158; (c) 66; (d) 2.729; (e) 2.147 
12.40 1^' - and Ц = 1 from Problem 12.31. 


12.41 For nonsingular А, 1 = ||I]| = ||AA "|| s ПАША ‘Il = (A). 


CHAPTER 13 
13.20 C, E, and F 
13.21 B,C, E, F, and H. 


13.22 Q, =(1/V2, 1/V2, 0], ©, =[-1/V6, 1/V6, 2/V6]’, and Q, = [1/V3, -1/V3, 1/V3]", corresponding 


to eigenvalues —2, —2, and 1, respectively. 


13.23 Q, = (1,0,0,0, 0]", Q, =[0, 1/V, 1/V2, 0, 0], Q, = [0, -1/V6, 1/V6, 2/V 6, 0]", О, = [0,0,0, 0, 1]", 
and Q, = (0, 1/V3, -1/V3, 1/V3, 0]’, corresponding to eigenvalues 3, 3, 3, 0 and 0, respectively. 


МЗ, -1/V3, 1/V3, 0]7, ©, = [i/V 3,0, -1/V3, U/V3]", Q, = [i//V 6, 2/V6, 1/V6, 0|7, and 
—i/ V6, 0, 1/V 6, 2/V 6", corresponding to eigenvalues 1, 1, 4, and 4, respectively. 


13.25 (а) Е cannot be reduced using only ЕЗ; (Б) three positive values and two zeros; (c) four positive values 


d CT 


13.27 (a) Eitself (Б) |[ -1 4-2 2+1 

16] 52116 62-190 72-1145 

-2-i16 #100 -16+ 90 

. [3 -i -[|-i -2 ET 

ma a[i a] we-4[77] e-D 3 0] 
13.29 (А + В)" =(A+B)" = (А + B' =A" - B' =A" +BY =A +B 


13.30 (AB)" = (AB)' = (BA)' = (BA)' = A'B' = АВ" = AB, implying that powers of Hermitian matrices are 
Hermitian, because A commutes with itself. 1 


13.31 Construct the vector E, so that it has a 1 as its kth component and all other components are zero. Then 
(AE,, E,) = a,,, which is real by Property 13.5. 


216 


13.32 


13.33 


13.34 


13.35 


13.36 


13.37 


13.38 
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AA" = A(-A) = (—A)A = A”"A 
(iA)" = (ТА)! = (iA)! = (iA)! = - iA? = - iA" = i(- A") = iA 
(AX, X) = (X, A*X) = (X, A"X) = (X, -AX) = - (X, AX) = - (AX, X) 


Let A be an eigenvalue of A corresponding to the eigenvector X. Then 
A(X, X) = (AX, X) = (AX, X) = (X, A*X) = (X, A"X) = (X, -AX) = (X, -AX) = -À (X, X) 
Thus, A 


—À, and A is pure imaginary. 
AOT uc Ч 


(A + A") is Hermitian and 1(A — A") is skew-Hermitian for any matrix A. For real A, these matrices 
are symmetric and skew-symmetric, respectively. 


According to (8.1), f(A) can be written as an (л — 1)-degree polynomial in A. Since the eigenvalues of A 
are real, so are the coefficients of such a polynomial. The result then follows from Problems 13.29 and 
13.30. 


CHAPTER 14 


14.17 


14.18 


14.20 


1422 


14.24 


14.25 


14.26 


14.27 


14.28 


14.29 


A, E and G are positive definite; B and D are positive semidefinite. 


1.6907 0.27849 -0.23915 ga xa 
A‘? =] 0.27849 1.6927 --0.23915 14.19 BY?=3) 2 2 -1 
-0.23915 -0.23915 2.2103 -1 -1 5 


id v3 0 0 
v2 =[ z 2| 1421 L-| vii У83 0 
че -УПЗ -М16ё V92 


3 0 0 
-1 22361 0 
0 1.3416 2.6833 
- 


—0.44721 —0.89443 


2 0 0 
L=|-i 3 0 1423 L- 
1 13 УЗ 


0 

0 

0 

2. 

((А + B)X, X) = (AX, X) + (ВХ, X) >0 

(АНХ, X) = (X, AX) = (АХ, X) = (AX, X) 50 

The eigenvalues of Аг ' are the reciprocals of the eigenvalues of А (Property 7.4) and are, therefore, 
positive. Furthermore, (A ')" = (A") '=A™', so A ' is Hermitian. A Hermitian matrix with positive 
eigenvalues is positive definite. 


For any X50, set Y=CX. Then Y#0 because Х = C 'Y, and (BX,X) = (C”ACX,X) = 
(ACX, CX) = (AY, Y) 20. 


В = At is Hermitian, so /(В) is Hermitian for any function (Problem 13.38); in particular, f(B) = e". If A 
is an eigenvalue of A, then At is an eigenvalue of B, and e*' > 0 is an eigenvalue of e" (Property 10.3). А 
Hermitian matrix with positive eigenvalues is positive definite. 


If (AX, X) is positive for all complex-valued vectors X, then it is also real for such vectors. It follows 
from Property 13.5 that A is Hermitian. 
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14.30 If 


1.2 
A- H 1 and = X=[x,,x,]” 


with X real and nonzero, then (AX, X) = x; + 2x,x, + xi = (x, + x,) >0. 
CHAPTER 15 
15.16 Cand E 


ANS) -9/97 1У2 -1V3 106 1v3 


m (b) 0 1 0 1УЗ 2Iv$ 
517 (а) [9//97 im) [л Е | © 0 
1/V2 0 1У2 ПУЗ -1/V6 1772 


a tt [9/97 АЛУ exits 13 
15.18 (a) win u= [97097 -9V7 we have U' ^u- [3 23) 


0 1 от о 0 3 -v3 -v$ 
(b) With vv 0 va: -vin vin] ме һауе — U"BU-|0 2 v2 
ПУ? 0 uvajlo УТЗ vin 0 0 4 


0 1/2 v3n 


1УЗ 2v6 0 [ 0 0 
1N3 -имМ6 1V3 006 


300 
co win v [ING ПУ uvi|o -v3m Al we have vrev-[0 2 o] 


0 = 
010 -2V3 1 


15.19 еш E (5) 
-4/5 3/5 1.0 3 
1 


0 Д (c) Al 2 9) 


15.20 ||UX|[? = (UX, UX) = (X. X) = |х| 
15.21 (UX, UY) охо, = Сх, Y) ЛХ ПУП 
15.22 Since R is both real symmetric and orthogonal, R^ = RR = R'R- В 'Е = 1. 


15.23 The eigenvalues are nonnegative (the matrix is Hermitian) and have absolute value 1 (the matrix is 
unitary), so all eigenvalues must be 1. 


15.24 Simply combine Properties 7.8 and 15.4. 


1 0 0 00 Y 0 0 0 0 
0 cos@ sind 0 0 0 cos0 0 -sinð 0 
1525 R,,=|0 -sin@ соё 0 0| R,=|0 0 1 0 0 
0 0 0 10 0 sind 0  cos8 0 
0 0 0 01 0 0 0 0 1 


15.26 Direct multiplication yields R7,(6)R,,(6) =I. 


15.27 The (k, p) element of the product is a,, cos 6 — a,, sin 6. Choose 0 to make this quantity equal to zero. 


CHAPTER 16 


16.11 (а) | 8: 1 1) (b) | 2 2 zl 
1 3-1 2 2-1 
-1 -1 5 -1 
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(с) 9 — 179 -3 (4) qot Zr 
ed 8$ uw 0 sp 3*2 
сз 9 =з 2 43 1! 
= O33 <6 A eF 1 
16.12 (a) and (c) are positive definite; (b) is positive semidefinite. 
16.13 (a) [1 0 0 (6) [10 0 (< [10 0 0 (d) о о 0 
010 010 0100 01 0 0 
ool 000 0010 00-1 0 
0001 00 O0 -1 
16.14 They are not congruent because they do not have the same inertia matrix. 
16.15 (a) Three positive eigenvalues; (b) two positive eigenvalues and one zero eigenvalue; (c) four positive 
eigenvalues; (d) two positive and two negative eigenvalues 
16.16 An identity matrix 
16.17 (а) 0 1 0 (6) 1 0 0 (с) 1 о 0 
2N1? -2//17? ANT -3/2 0 1/2 -8/9 1/9 1 
1/2 e| 0 z BE us: e t 
10 0 1 0 0 
16.18 Inertia matrix -|O 1 0 P=} -(1 + (2) 1 0 
00-1 (7+110)/5 -(5S-i)/5 1/5 
16.19 Given А=Р,ВР/ and В=Р,СР/. Set P,=P,P,. Then A = Р,ВР = P,(P,CP7)P7 = (Р,Р,)С(Р,Р,)' = 
P,cP{ 
16.20 If A= PBP” then В = ФАО”, where Q=P 
16.21 Denote as N an inertia matrix congruent to A. Then A is also congruent to N (Problem 16.20). (a) If B is 
congruent to A and A is congruent to N, then B is congruent to N (Problem 16.19). (b) If both A and B 
are congruent to N, then A is congruent to N and N is congruent to B so A is congruent to B. 
16.22 The eigenvalues of A ' are the reciprocals of the eigenvalues of A, and therefore have the same signs. It 
follows from Property 16.1 that A and A^' are congruent to the same inertia matrix and to themselves. 
CHAPTER 17 
17.20 Reducible, not primitive, not stochastic, 3« ø «4 
17.21 Irreducible, primitive, not stochastic, 3 0 —4 
17.22 Irreducible. not primitive, not stochastic, 1 = o <2 
17.23 Irreducible, primitive, not stochastic, с = 4 
17.24 Irreducible, primitive, not stochastic, с = 7 
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17.25 Irreducible, primitive, stochastic, ergodic, o = 1, 
19/45 17/45 9/45 
=| 19/45 17/45 9/45 
19/45 17/45 9/45 
17.26 Reducible, not primitive, stochastic, ergodic, с = 1, 
100 
L-|10 0 
100 
17.27 Reducible, not primitive, stochastic, ergodic, с = 1, 
3/8 0 5/8 
L=|0 1 0 
3/8 0 5/8 
17.28 Reducible, not primitive, stochastic, not ergodic, с = 1, no limit exists. 
17.29 Irreducible, primitive, doubly stochastic, ergodic, с = 1, 
M3 1/3 1/3 
L=/1/3 1/3 1/3 
1/3 1/3 1/3 
17.30 (a) 56.48 percent; (b) 55.56 percent 
17.31 Probabilities аге 0.2, 0.14, 0.154, 0.151, and 0.15162. 
17.32 (a) Approximately 34 discharged, 31 ambulatory, 18 bedridden, and 17 dead; (b) approximately 65 
discharged and 35 dead 
CHAPTER 18 
18.12 (a) Band matrix of width three, tridiagonal, Toeplitz, and Hessenberg; (5) circulant and Toeplitz; (c) 
Hessenberg; (d) circulant and Toeplitz; (e) tridiagonal, Toeplitz, and Hessenberg; (f) none; (g) 
tridiagonal and Hessenberg 
18.13 A,=A,=2, A,=A,=0, with eigenvectors [1,1,1,1]", [1,-1,1,-1]’, (1,4 1, ~-i)", and 
[1, —i, -1, 47, respectively 
18.14 A, 26, А, = (-3- iV3)/2, and А,=(—3 + iV3)/2, with eigenvectors [1,1,1], [1,(-1 + iV3)/2, 
(71- iV3)/2]", and [1, (-1— iV3)/2, (—1 + iV3)/2]", respectively 
18.15 4.46410, 1, —2.46410 
1836 [1 0 0 12 0 18.17 [-1 0 0 0 Y 2 © 0 
3 -5 0 |01 -2/5 25 0 о Jo 115 0 
0 з mn/sAo 0 1 0 2 -12/5 0 0 0 1 -5/12 
00 1 -71240 0 0 1 
1 0 0 200 
18.18 WithU=|0 -1/V2 -1/V2 ме һауе | U'BU-|-V2 4 0 
0 -uv2 1v3 033 
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18.19 With U as in Problem 18.18, we have 


3 -v2 0 
UCU-|-2 51 
0 13 
1 о о оү 000 -v3 0 0 
= г 2 
ua жшше ФУ ШР Leta wawl 22 12 8 
0 v3 0 vžilo о 1 0 0 và -i 
06 08 0 232 276 120 
18.21 With R,,(0.927295) -| -0.8 0.6 O| wehave — RLAR,-|-124 0.68 -1.60 
0 01 0 5.00 1.00 


18.22 First iteration (k 4, i= 1, j =2): 


0.554700 -0.832050 0 0 
к,,(—0.9в279а) = | 0892050 0.554700 0 O| and A becomes 
0 o 01 
2.153846 -1.230769 4.160051 0 
-1.230769 —1.153846 —0832050 7.211103 
4.160051 -0.832050 2 1 
0 721103 1 E 
Second iteration (k = 4, i72, j=3): 
1 0 о о 
0 0.137361 0.990521 0 
В,,(1.433000) = 0 -0.990521 0.137361 0 and A becomes 
0 о 0 1 
2.153846 -4.289876 -0.647648 0 
-4.289876 2.166909 0.37154 0 
-0.647648 0.371544 -1.320755 7.280110 
0 0 72800 — -1 
Third iteration (k = 3, i= 1, j =2): 
0.497612 -0.867400 0 0 
| 0.497 
R,(-1049953)- 080400 0.492612 0 0) and A becomes 
0 0 01 
1.539591 2.171016 0 0 
2.171016 5.860346 0.746655 0 
O 0.746655 -1.320755 7.280110 
0 0 7280110 ~1 
5 -4 15 -12 
|а =з 12 -9 
зз сәр=|5 2 2 79 
4 -3 0 -9 
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18.24 No solution 18.25 sf zd 1826 X-[1,-1,0] 


CHAPTER 19 
19.13 
Iteration Eigenvector components Eigenvalue 
1.0000 1.0000 1.0000 
1.0000 0.7778 0.7778 9.0000 
1.0000 0.7183 0.7746 7.8889 
1.0000 0.6887 0.7746 7.8732 
1.0000 0.6737 0.7746 7.8730 
1.0000 0.6661 0.7746 7.8730 
19.14 
Iteration Eigenvector components Eigenvalue 
1.0000 1.0000 1.0000 
—0.7143 —0.2857 1.0000 7.0000 


—0.7143 —-0.2857 1.0000 —11.0000 
—0.7143 -0.2857 1.0000 = 11.0000 


The eigenvalue is 14 + (711) = 3. 


19.15 
Iteration Eigenvector components Eigenvalues 
0 1.0000 1.0000 1.0000 
1.0000 -0.5313 0.2188 32.0000 
1.0000 0.2960 0.1038 30.4063 
1.0000 -0.2102 0.1613 29.1192 
1.0000 0.1040 0.1253 29.8552 
19.16 


Iteration Eigenvector components Eigenvalue 


—2.0000 
—2.0000 


19.17 The eigenvalues are —4, —2, and 1, with corresponding eigenvectors V, = [1, 0, —1]", V, = (1,0, 1]’, and 
У, = [0, 1, 0]”, respectively. With X, = [1, 1, 1]", we have X, = 0У, + 1V, + 1V,; thus, X, is a linear 
combination of V, and V,, with no component that is influenced by V,, the eigenvector corresponding to 
the dominant eigenvalue. 
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19.18 There is no convergence, implying that the dominant eigenvalue is complex. 


19.19 
Iteration Eigenvector components Eigenvalue 
1.0000 1.0000 1.0000 А 
0.9697 0.6970 1.0000 33.0000 
0.9591 0.6899 1.0000 30.3939 
0.9578 0.6891 1.0000 30.3011 
0.9577 0.6890 1.0000 30.2902 
0.9576 0.6889 1.0000 30.2889 
0.9576 0.6889 1.0000 30.2887 
19.20 3 " i 
Iteration Eigenvector components Eigenvalue 
0.5774 0.5774 0.5774 
0.6448 -0.0586 0.7621 7.6667 
0.7216 0.0147 0.6922 11.8454 
0.7034  —0.0037 0.7108 11.9902 
0.7080 0.0009 0.7062 11.9994 
0.7069 -0.0002 0.7073 12.0000 
0.7072 0.0001 0.7070 12.0000 
19.21 Е " 
iteration Eigenvector components Eigenvalue 
0.5774 0.5774 0.5774 
—0.0765  —0.9941 0.0765 -4.3333 
0.1541 0.9760 -0.1541 -13.3158 
—0.2282  -0.9465 0.2282 — 13.6491 
There is no convergence yet 
19.22 


Iteration Eigenvector components Eigenvalue 


1.0000 1.0000 1.0000 
1.0000 0.0000 — —1.0000 


1.0000 0.1389 —0.7778 
1.0000 0.1417 -0.7746 
1.0000 0.1418 -0.7746 


The eigenvalue is 1/7.8732 = 0.1270. 


19.23 There is по unique solution to LZ = Х; hence A = 0 is an eigenvalue 
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19.24 


Eigenvector components Eigenvalue 


1.0000 1.0000 1.0000 1.0000 
—0.1765 1.0000 | —0.6353  —0.7059 
0.3041 1.0000 0.1596 0.0851 
0.1405 1.0000 -0.0661  —0.0968 


19.25 A=9+ 1/3.08114 = 9.3246 
19.26 A=2.5 + 1/39.5709 = 2.5253 


19.27 First iteration: shift = 7.66667; eigenvector = [1, —0.486452, 0.89935]" 
Second iteration: shift = 10.5092; eigenvector = [0.976710, 0.067597, 1]” 
Third iteration: shift = 11.969339; eigenvector = [1, —0.000168, 0.999924)” 
Fourth iteration: shift = 12.0000; eigenvector = [1, 0, 1]” 


CHAPTER 20 


-0.6667 0.2357 0.7071 
20.14 (a) Q-| 0.6667 –0.2357 0.7071 


6  -5  -0.6667 
0 2.8284 0.2357 
0.3333 0.9428 0 


0 0 2.1213 


0.7428 0.6442 -0.1826 5. a 3.1568 —2.0426 
(b) О=| 0.5571 -0.7459 -0.3651 1.0171 —0.5425 
—0.3714 0.1695 –0.9129 0 0.7303 


(c) The third column is a linear combination of the previous two, so Step 20.1” must be employed for Q: 


0 -1v3 ia 1 0 1 
Q=|-1 0 R=|0 V2 0 
0 -1У2 -w3 ооо 
0.9057 —0.2959 –0.3037 27.6043 —139.0361 - 85.0591 
(d) Q-| 0.3985 0.8387 0.3712 0 17.1747 10.3479 
-0.1449 0.4572 -0.8775 0 0 19.4391 
0 -v22 0 vin P 1 a 
-1 0 0 0 _|0 0 vin 
© 97| o -v32 o -v32| Flo o 1 o 
0 0 -1 0 0 0 0 v27 


0.6644 -0.3332 0.6606 —0.1055 
0.7474 02962 -0.5872 0.0938 
0) 9-| `o -00254 0.1452 0.9891 
O 08948 -0.4446 00423 


1,324.5754 493.2902 904.3653 246.6451 


| ° O ^ 09835089 7,413.9901 98.3798 
| d O 4,286070 -197.4178 
0 0 0 62315 


20.15 10.1461, 5.92049, —0.0665892 20.16 10.6056, 3.39445, 3 
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20.17 3.41421, 2, 0.585786 — 20.18 1, —17 + i24, –17 – i24 
20.19 3.61803, 2.61803, 1.38197, 0.381966 20.20 990, 660, 440, 330 
20.21 9,9,9,9 20.22 -1+iV3, +i 
20.23 The QR algorithm does not converge. 20.24 232.275, 79.6707, 63.8284, 24.2261 
CHAPTER 21 
21.18 [1/2, 1/2] 21.19 fil 21.20 [fi 1 1 21.21 [ -2 J 
РУ] E I 9 1 dl E 
33 tat 
21.22 ali 4 ~ ] 21.23 26 -23 21.24 1 Т 
914 -2 5 1 36 -16 x 1 1 
э! 38 6 1 =1 0. 
206) 16 -30 
-6 3 
21.25 x,=3/22, x,=3/22, x,=9/22 24.26 x, = x, =x, =2/3 
21.27 x,=2/3,x,=5/3 21.28 x, =3/206, x, = 201206, x, = 441206, x, = —14/206, x, = 31/206 
21.29 Form ||AX - B||}, and then set the first partial derivatives with respect to each component of X equal 
to zero. 
21.30 A"AX = (QR)"(QR)X = R"(Q"Q)RX = R"IRX = R"RX 
and A"B = (QR)"B = R"Q"B 
When the columns of A are linearly independent, the diagonal elements of R are all nonzero, and 
(В)! exists. 
—0.566947 0.377964 
—0.377964 0.377964 
—0.188982 0.377964 
21.31 Solve RX = Q"B with Q= O 0.377964 [^ Я а] 
0.188982 0.377964 н ü 
0.377964 0.377964 
0.566947 0.377964 
0.447214 0.596285 
21.32 Solve RX = Q"B with e [ro 290] в [22508 102и 
0 0.745356 К 
21.33 The normal equations are 


354a + 100b + 30c = 1158 
100а + 30Ь + 10с = 342 
30а + 106+ 5с= 122 


and the solution is а = 2.57, Б = —0.69, с = 10.54. 
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21.34 Vii -uv2 32 Wi 175 0 00 


[уп М? m о-у "m з ? 0] 
ЗУП 0 -2V2 


0.597354 соотв) =0 [ 
1" | -0.534078 –0.845435 


—0.845435 0.534078 p= [5294 0 ] 
0.801978 0.597354 0 


2135 V-V, =[ 0.444992 
21.36 A satisfies Properties I1 through I3 because 
(ААТ!) 21" =1= AA"! 
ААТА = A(A A) = АІ = A 
and AAA’ = (A`'A)A™' =A = At 


The result then follows from Property 21.1. 
21.37 A” =0 satisfies conditions I1 through I3 when А =0. 


21.38 Conditions I1 through I3 are symmetric with respect to A and A‘. Thus, if A* is the generalized inverse 
of A, then A is also the generalized inverse of А“. That is, А = (А*)'. 


21.39 Show that conditions 11 through 13 are satisfied. 
2140 It follows from Property 21.5 that (A*)" = (A")* = А". 


21.41 Conditions I1 through I3 are satisfied because 
(A'A)" = (АА) = A" = А = АА = A*A 
A'AA = AAA = A(AA) = AA =A 

and А*АА* = AAA = А(АА) = AA =A = A* 

21.42 AA* and A*A are Hermitian from condition I1. Also, 
(AA*)(AA*) =(AA*A)A* = АА” апі —(A'A)(A'A) = (A'AA*)A = A*A 

21.43 Take X to be the n x n diagonal matrix containing all the singular values of A, including zeros if they 

arise. Construct D, V, and U, exactly as described in Steps 21.8 and 21.9, and then construct U by first 


following Step 21.10 but then keeping only the first n linearly independent columns and orthonormaliz- 
ing them. 


21.44 Р is Hermitian and similar to F, which has nonnegative eigenvalues. Since the columns of аге 
orthonormal, U” U = I, and, with V being unitary, 


M"M = (UV^)"(UV^) = VU"UV" = VIV" - vv" - 1 
2145 MP=(UV")(VEV") = (УУ) ХУ" -UXV"- A 


21.46 With V and U as given in Problem 21.4 and 


we have P - VE V" =A and М = UV" =I. 


21.47 With D, V, and U, as given in Problem 21.5, we have X = D and U = U,. Then P- VEV" = X = D and 
M-UvV"-U-U, 
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Addition of matrices, 2 Diagonal, 24, 127 
Adjoint, 119 Diagonal matrix, 24, 127, 137 
Angle between vectors, 143 Diagonal quadratic form, 144 
Augmented matrix, 11 Differential equations, 72 
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Distribution vector, 153 

Band matrix, 160 Dominant eigenvalue, 111, 152, 169 
Bessel function, 80 Dot product, 1 
Block matrix, 43 Doubly stochastic matrix, 153 
Canonical basis, 82 е^“ (matrix exponential), 72, 100 

for a normal matrix, 119 for a positive definite matrix, 135 
Cayley-Hamilton theorem, 61 Eigenvalue, 60 
Chain, 82 bounds on, 112 
Characteristic equation, 60 of a circulant matrix, 160 

of similar matrices, 91 dominant, 111, 152, 169 
Characteristic polynomial, 60 of an elementary reflector, 143 
Characteristic value (see Eigenvalue) of f(A), 93 
Characteristic vector (see Eigenvector) of a Hermitian matrix, 119 
Cholesky decomposition, 129 of an inverse, 60 
Circulant matrix, 160 of a nonnegative matrix, 152 
Coefficient matrix, 11 by numerical methods, 169-170, 181 
Cofactor, 42, 50 of a positive definite matrix, 128 
Cofactor matrix, 44 of a positive matrix, 152 
Column rank, 53 by QR algorithm, 181 
Column vector, 1 of similar matrices, 91 
Companion matrix, 70 of a singular matrix, 60 
Compatible norms, 111, 116 of a transpose, 60 
Complete pivoting, 19 of a triangular matrix, 60 
Complex conjugate, 103 for a unitary matrix, 136 
Complex conjugate transpose matrix, 119 Eigenvalue rank number, 82 
Complex matrix, 1, 103 Eigenvector, 60 
Complex quadratic form, 144 of a circulant matrix, 160 

(See also Quadratic form) generalized, 82 
Component, 52 
Condition number, 118 maximal linearly independent set of, 61 
Congruent matrices, 144, 151 of a nonnegative matrix, 152, 153 
Conjunctive matrices, 145 of a normal matrix, 119 
Consistency condition for norms, 110 by numerical methods, 169, 170 
Consistent simultaneous equations, 11 of a real symmetric matrix, 119 
Constant matrix, 1 Elementary column operations, 3 
Convex combination, 59 Elementary matrix, 24 
Crout's reduction, 25 determinant of, 43 

for a tridiagonal matrix, 161 inverse of, 34 

Elementary reflector, 137 

Derivative of a matrix, 72 Elementary row operations, 3 
Determinant, 42 Equality of matrices, 1 

and eigenvalues, 60 Ergodic matrix, 153 

of an LU decomposition, 51 Euclidean inner product, 103, 120 

of a positive definite matrix, 132 Euclidean matrix norm, 111 

for a unitary matrix, 136 Euclidean vector norm, 110, 116, 194 
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Expansion by cofactors, 42 
Exponential of a matrix, 71 


Finite Markov chain, 153 

Frobenius norm, 111, 116 

Function of a matrix, 71, 92 
eigenvalues of, 93 
Hermitian, 127 


Gaussian elimination, 12 
Gauss-Jordan elimination, 21 
Generalized eigenvector, 82 
Generalized inverse, 192 
Gerschgorin's theorem, 170 
Given's method, 167 


Gram-Schmidt orthogonalization process, 104, 121 


modified, 181 


Hermitian congruent matrices, 145 
Hermitian matrix, 119 
eigenvalues of, 119 
functions of, 127 
Beneralized inverse of, 192 
positive definite, 128 
for a quadratic form, 144 
Rayleigh's quotient, 146 
Hermitian transpose matrix, 119 
for a unitary matrix, 136 
Hessenberg form, 161, 182 


Homogeneous linear equations, 12, 23, 55, 59 


Householder transformation, 137 


Idempotent matrix, 33, 202 
Identity matrix, 24, 118 
Inconsistent simultaneous equations, 11 
Index, 145 
of a nilpotent matrix, 32 
Induced norm, 111 
Inertia matrix, 145 
Infinite series of matrices, 71 
Inner product, 103, 136, 144 
Integral of a matrix, 72 
Inverse, 34, 44 
of a circulant matrix, 160 
and congruence, 151 
determinant of, 43 
eigenvalue of, 60 
of an elementary matrix, 34 
of an orthogonal matrix, 136 
by row reduction, 34 
of a unitary matrix, 136 
Inverse power method, 170 
modified, 180 
Invertible, 34 
Irreducible matrix, 152 


Jordan block, 91 
Jordan canonical form, 91 


Kronecker product, 165 


1, norm, 110 

l, norm, 110 

1, norm, 110 

1, norm, 110 

L, norm, 111 

L. norm, 110 

Least squares, 194, 201 

Left eigenvector, 67, 154 

Length, of a chain, 82 
of a vector, 110, 143 

Linear combination, 52 
of eigenvectors, 66 

Linear dependence, 52, 53 


Linear equations (see Simultaneous linear 


equations) 

Linear independence, 52, 53 
of a chain, 85 
of eigenvectors, 61 

Lower triangular matrix, 24 
determinant of, 42 
eigenvalues of, 60 
inverse of, 32 

LU decomposition 24, 40, 51 
Cholesky decomposition, 129 
Crout's reduction, 25, 161 


Magnitude of a vector, 110 

Main diagonal, 24 

Markov chain, 153 

Matrix, 1 

Matrix equation AX + XB = C, 73 


AX = B (see Simultaneous linear equations) 


AXB = C, 166 
Matrix norm, 110 


Maximal set of linearly independent vectors, 57 


eigenvectors, 61 
Minimum polynomial, 83 
Minor, 42 

principal, 128 
Modal matrix, 91 
Modified Gram-Schmidt process, 174 
Moore-Penrose inverse, 192 
Multiplication of matrices, 2, 6 


Negative definite matrix, 128 
Negative semidefinite matrix, 128 
Nilpotent, 32 
Nonhomogeneous linear equations, 12 
(See also Simultaneous linear 
equations) 
Nonnegative matrix, 152 


Nonsingular matrix, 34 
Nontrivial solutions, 12 
Norm, 110 
Normal equations, 201 
Normal matrix, 119, 136 
similar to a diagonal matrix, 137 
Normalized vector, 110 
Null space, 59 
Numerical methods for eigenvalues, 169, 170, 181 


Order, 1 

Orthogonal matrix, 136, 137 
Orthogonal vectors, 103 
Orthonormal vectors, 110, 119, 136 


Partial pivoting, 13 

(See also Pivoting strategies) 
Partitioned matrix, 5 

determinant of, 43 
Permutation matrix, 152 
Perpendicular (see Orthogonal vectors) 
Perron's theorem, 152 
Perron-Frobenius theorem, 152 
Pivot, 3 
Pivotal condensation, 43 
Pivoting strategies, 12, 18, 19, 25, 35 
Polar decomposition, 202 
Positive definite matrix, 128 
Positive matrix, 152 
Positive semidefinite matrix, 128 
Power method, 169 

modified, 174 
Powers of a matrix, 26 
Primitive matrix, 153 
Principal diagonal, 24 
Principal minor, 128 
Pseudoinverse, 192 


QR algorithm, 181 
QR decomposition, 181, 201 
Quadratic form, 144 


Rank, of a generalized eigenvector, 82 
of a matrix, 3, 12, 43, 53, 144, 192 
Rank number of an eigenvalue, 82 
Rayleigh’s principle, 146 
Rayleigh’s quotient, 146, 174, 180 
Real matrix, 1 
Real symmetric matrix, 119, 161, 174 
for a quadratic form, 144 
Reducible matrix, 152 
Regular matrix, 153 
Right eigenvector, 60 
(See also Eigenvector) 
Rotation matrix, 143, 167 
Row rank, 3, 53 
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Row vector, 1 
Row-echelon form, 2 
reduction to, 3 


Scalar, 1 
Scalar multiplication, 2 
Scaled pivoting, 18 
Schur decomposition, 136 
Schwarz inequality, 103 
Self-adjoint matrix, 120 
Sequences of matrices, 71 
Series of matrices, 71 
Shifted inverse power method, 170 
modified, 180 
Shifted QR algorithm, 182 
Signature, 145 
Similar matrices, 91, 123 
Similarity transformation, 137 
QR algorithm, 188 
Simultaneous linear equations, 11, 25, 35 
homogeneous, 12, 55, 59 
matrix form, 11 
(See also Solutions of simultaneous linear equa- 
tions) 
Singular matrix, 34 
eigenvalues of, 60 
Singular value, 193 
Singular value decomposition, 193 
Skew-Hermitian matrix, 127 
Skew-symmetric matrix, 127 
Solutions of simultaneous linear equations, 11 
by Gaussian elimination, 12 
by Gauss-Jordan elimination, 21 
by inversion, 35 
least squares, 194 
by LU decomposition, 25 
theory of, 12, 23, 55 


trivial, 12 
Spectral norm, 111, 141 
Spectral radius, 111, 118 


for nonnegative matrices, 152 
Square matrix, 24 
Square root of a matrix, 128 
Stochastic matrix, 153, 158 
Subdiagonal, 24 
Subtraction of matrices, 2 
Superdiagonal, 24 
Sylvester's law of inertia, 145 
Symmetric matrix (see Real symmetric matrix) 


Toeplitz matrix, 160, 161 
Trace, 60, 70, 91, 172 
Transpose, 1 
of a band matrix, 160 
determinant of, 43 
eigenvalues of, 60 


Transpose (continued) 
inverse of, 35 
of an orthogonal matrix, 136 
rank of, 53 
of a right eigenvector, 67 
(See also Hermitian matrix) 


idiagonal matrix, 161 
Trivial solution, 12 


Unit vector, 110 

Unitarily similar, 137 

Unitary matrix, 136 

Unitary transformation, 136 
preservation of angles, 143 
preservation of length, 143 


Upper triangular matrix, 24, 136 
determinant of, 42 
eigenvalues of, 60 
inverse of, 34 
when normal, 124 


Vandermonde determinant, 66 
Vector, 1, 52 

angle between, 143 
Vector norm, 110 


Well-defined function, 71 
(See also Function of a matrix) 
Work column, 3 


Zero matrix, 2, 53, 192 


